The Problem
When using, cloudbuild.yaml files, specifically named for their build environment, such as cloudbuild-dev.yaml and cloudbuild-prod.yaml, and configured/targeted in the Trigger settings they aren't found/recognized when GCB reacts to a GitHub event (push etc).
However, it's working just fine when manually running the Trigger from GCB console.
When using an ordinarily named cloudbuild.yaml, in the root of the project, Cloud Build correctly runs the expected steps.
The Workaround
In short, there isn't an easy one (imo). But to get it run you need to use just a single _cloudbuild.yaml).
However, to effectively re-use that for both dev and prod environments one is blocked by this issue
Related
I have a repository on GitLab with a directory structure similar to this:
folder-a\
-python-a.py\
folder-b\
-python-b.py
I am trying to set up a CI/CD pipeline on gitlab that will detect changes made to the python code, and deploy them to a production server. What I have currently is the user have to trigger the pipeline manually, and input in the folder name as a variable, which will then cause the pipeline to "cd" into the folder and deploy the code inside the folder.
Is there any configuration or settings that can be added to the pipeline so whenever a Merge Request is merged to the main branch, the pipeline triggers and detects which code was changed, and then deploy the respective code without having the user to manually trigger it and inputting the folder name as a variable?
You might be able to use only:changes / except:changes to do that.
You can have two jobs. One job that goes to folder-a if something under folder-a/* has changed and the other job goes to folder-b if something under folder-b/* has changed.
Currently we're having a dev environment in a gcp project. We're using GDM templates and other stuffs along with repo in bitbucket. Whenever we push any changes in bitbucket it builds and deploy to this dev environment. Suddenly, we've decided to have a new gcp project as test environment and we want to deploy automatically to this environment like dev environment. Our preference will be to deploy to this environment from the cloudbuild execution in dev environment. Can you suggest us any guideline that'll help us to set up things in one place that'll automatically deploy this in multiple projects as multiple environments automatically?
You can use Terraform to achieve this.
There's a lot of information on how to start here.
However, I would suggest having projects in separate deployments. This way you limit the blast radius and protect production from errors occurring in other environments.
You need separate calls for separate projects. Just like almost all Google API resources deploymentmanager/deployments lives inside a project (https://www.googleapis.com/deploymentmanager/v2/projects/[PROJECT]/global/deployments), thus you cannot deploy to multiple projects in one call.
I want to automate the deployment of Cloud Function in to Production project through Cloud Build whose source files are present in Cloud Source Repository of DEV project. How can I ensure that the moment I push the code in production branch of Cloud Source Repository of DEV project, the Cloud Function gets created in to Production Project .
If I understand, you are trying to trigger a build from a repository stored on another project.
This is not possible, the build triggers must be on the same project than the repositories
I think my answer will help here: How to pass API parameters to GCP cloud build triggers
Basically what Claudio recommended, use the examples to build your steps. I believe what you want to do is create a step that the. Triggers the cloud function when you push changes to the dev production branch. When the trigger is called and ran, you then add a step to either run the cloud function or use the REST API to trigger the build by its ID. See my example above.
I have a single Cloud Source Repository with multiple projects. I am able to create a cloudbuild.yaml file in the repo root that builds all projects. However, I don't want to have a build trigger that rebuilds all of the projects since most commits will be for a single project. Ideally I would like to have a cloudbuild.yaml file in each project subdirectory and a build trigger that detects changes in the project subdirectory of the repository. Is something like this possible?
As a possible workaround, I believe I may be able to keep my cloudbuild.yaml in the repository root and create a custom step that will get the commit sha (via the COMMIT_SHA substitution) and then get the list of files committed (via "git show --name-only --pretty=format: $COMMIT_SHA") to determine which project should be built and what image should be created. An alternative may be to have a tagging naming convention that will contain the project name and basing the trigger on that but I don't want to tag each commit.
Note, it seems like build triggers work very well when you have multiple repos but we made the decision to go with a mono repo and I don't want to rehash that debate in this question. I'd like to understand how to best use the Build Triggers in a mono repo.
When I trigger via Jenkins (code deploy plugin), I get the following error -
No such file or directory - /opt/codedeploy-agent/deployment-root/edbe4bd2-3999-4820-b782-42d8aceb18e6/d-8C01LCBMG/deployment-archive/appspec.yml
However, if I trigger deployment into the same deployment group via code deploy directly, and specify the same zip in S3 (obtained via Jenkins trigger), this step passes.
What does this mean, and how do I find a workaround to this? I am currently working on integrating a few things and so, will need to deploy via code deploy and via Jenkins simultaneously. I will run the code deploy triggered deployment when I will need to ensure that the smaller unit is functioning well.
Update
Just mentioning another point, in case it applies. I was previously using a different codedeploy "application" and "deployment group" on the same ec2 instances, and deplying using jenkins and code deploy directly as well. In order to fix some issue (not allowing to overwrite existing files due to failed deployments, allegedly), I had deleted everything inside the /opt/codedeploy-agent/deployment-root/<directory containing deployments> directory, trying to follow what was mentioned in this answer. However, note that I deleted only items inside that directory. Thereafter, I started getting this error appspec.yml not found in deployment archive. So, then I created a new application and deployment group and since then, I am working on it.
So, another point to consider is whether I should do some further cleanup, if the jenkins triggered deployment is somehow still affected by those deletions (even though it is referring to the new application and deployment group).
As part of its process, CodeDeploy needs to reference previous deployments for Redeployments and Deployment Rollbacks operations. These references are maintained outside of the deployment archive folders. If you delete these archives manually as you indicate, then a CodeDeploy install can get fatally corrupted: the references left to previous deployments are no longer correct or consistent, and deploys will fail.
The best thing at this point is to remove the old installation completely, and re-install. This will allow the code deploy agent to work correctly again.
I have learned the hard way not to remove/modify any of the CodeDeploy install folders or files manually. Even if you change apps or deployment groups, CodeDeploy will figure it out itself, without the need for any manual cleanup.
In order to do a deployment, the bundle needs to contain a appspec.yml file, and the file needs to be put at the top directory. Seems the error message is due to the host agent can't find the appspec.yml file.