I have a single Cloud Source Repository with multiple projects. I am able to create a cloudbuild.yaml file in the repo root that builds all projects. However, I don't want to have a build trigger that rebuilds all of the projects since most commits will be for a single project. Ideally I would like to have a cloudbuild.yaml file in each project subdirectory and a build trigger that detects changes in the project subdirectory of the repository. Is something like this possible?
As a possible workaround, I believe I may be able to keep my cloudbuild.yaml in the repository root and create a custom step that will get the commit sha (via the COMMIT_SHA substitution) and then get the list of files committed (via "git show --name-only --pretty=format: $COMMIT_SHA") to determine which project should be built and what image should be created. An alternative may be to have a tagging naming convention that will contain the project name and basing the trigger on that but I don't want to tag each commit.
Note, it seems like build triggers work very well when you have multiple repos but we made the decision to go with a mono repo and I don't want to rehash that debate in this question. I'd like to understand how to best use the Build Triggers in a mono repo.
Related
My plan is to create a build project with two sources:
The primary source is the repository of the application I'm building.
A secondary source is a repository containing a generic buildspec.yml and other files like eslint configs.
The reason for that is I want to separate app code from build definition, so I can reuse the same build definition for several apps. The same buildspec.yml and accompanying eslint files are capable of building any of my backend applications.
According to documentation the buildspec path for a build project is a path relative to the root path of primary source.
But what is the correct way of pointing to a buildspec.yml residing in secondary source?
(As far as I know the application code must be the primary source so Codebuild can detect code changes like PRs opened and code push operations.)
(I know that Codebuild allows a S3 path as buildspec path but I don't see how it can help me since my secondary source is a repository.)
Thanks!
My understanding is that you want ONE buildspec that you can reuse for multiple projects with a similar build. If this is the case, I think you can do this but you need to reverse your primary and secondary sources.
When you create a build project you have to define the buildspec that is going to be used. Your build project is going to use the buildspec from your PRIMARY source. So the SOURCE that has your primary buildspec will need to be your PRIMARY and the project you are going to build will be your SECONDARY.
Then, in your buildspec you can reference the commands pointing to your SECONDARY source using environment variables. In your buildspec, you would reference CODEBUILD_SRC_DIR_sourceIdentier.
I have done this with codepipeline having multiple sources. If you define your output for your secondary source to be called SECONDARY_SOURCE_OUTPUT
Then you would refer to it in your buildspec as $CODEBUILD_SRC_DIR_SECONDARY_SOURCE_OUTPUT/.
Your buildspec exists in your Primary source but you would set your commands would execute from the directory above in your buildspec. Now you can have several similar projects that have the same build pattern use the same buildspec.
In the case below, I use the same buildspec from the project on the left in the source stage, with different projects that can be pulled in as secondary sources.
This link has some information on multiple sources.
We are using Build Pipeline in Azure DevOps to create a Deployment Artifact. Typical steps in such pipeline are:
Build Solution / Project
Copy dlls output into $Build.ArtifactStagingDirectory
Publish Artifact from $Build.ArtifactStagingDirectory
I just wonder if I can rely on the fact, that on start of each Build the Build.ArtifactStagingDirectory is empty. Or should I clean the folder as first step to be sure?
From my experience the folder was always empty, but I am not sure if I can rely on that. Is that something specific to Azure hosted Agent and maybe by using custom Build agents I have to do manual clean-ups of this folder? Maybe some old files from last build could remain there? I did not found this info in documentation.
Thanks.
I think that the main idea of this variable $Build.ArtifactStagingDirectory is to be a clean area so you can manage the code you're pushing from your repo. As far as I know, there is no explicit information on documentation talking that this folder is empty at every new build, but there are a few "clues":
You can see at the Microsoft's Build Variables documentation that Build.StagingDirectory is always purged before each new build, so you have a fresh start every build.
In the documentation above you have a few cases where it explicitly cites that some folders or files are not cleaned on a new build, like the Build.BinariesDirectory variable.
I've run a few build and realeases pointing to my Web App on Azure, and I never saw an unwanted file or folder that was not related to my build pipeline.
I hope that helps.
We have a microservice project architecture where there is a single project repository with several folders. Each folder has files etc for a specific API. We would like to have that as a single repo but configure separate jobs in jenkins for each API folder. As such we would like to know how to use same repo for scm checkout in jenkins but trigger builds for commits made only to the folders where changes are made. i know it supports regex to include and exclude. But would like to know how best to use that.
So say for example i have a project sample-project with 3 folders abc, def and xyz.
we now have a job in jenkins that checkouts sample-project. Now we would like that jenkins job to be configured in a way that only when anything inside abc folder is changed or committed, it triggers that job otherwise not. How to best implement this.
Im trying to figure out how to chain multiple 'promotions' (by a user clicking) whilst ensuring that ever build in the chain is not queued. By current setup is as follows, NOTE as my application is a white label the configuration described below is repeated for every site.
Build & Test - Creates zipped artifact
Deploy to Testing - Has artifact and snapshot dependency
Deploy to Staging - Has artifact and snapshot dependency
Deploy to Production Has artifact dependency
When promoting to production i want to do this across all websites (without having to manually click promote on each build).
I am currently trying the following strategy, to set the 'deploy to production' build to have a Artifact dependency, without a snapshot dependency so it doesn't queue down the chain. I have set the artifact to depend on the 'Build & Test' configuration to gain access to the zipped project and i have set it to build with a specific build number referencing a parameter in the production build.
After doing some googleing i found out that i am able to get the stagings build number using the rest api as follows:
http://teamcity_url/httpAuth/app/rest/builds/buildType:build_configuration_id/resulting-properties/build.number
And this works great, however i don't understand how i can get this value into the parameter?
Also i dont know if my approach is correct? is there a better way?
Set up the artifact dependencies chronologically (Build -> Test -> Staging -> Production) and all your snapshot dependencies to Build & Test. Depending on exact needs you might have a snapshot dependency on both Build and the one your artifact dependency is on.
Also make sure you enable "Do not run new build if there is a suitable one. This should keep it from queuing down the chain without intention.
Using the build chain tab will be important because the main project page only shows the last build ran. So clicking run from there will que the chain because you are asking for a new build, even though to you it might feel like your asking for the next step to be ran. The build chain tab helps keep things clear.
Aim: Set up an ant/cmd script that will package the artifacts from several builds into a single zip. I plan to do this by setting up a final build configuration that will have a dependency on those several projects.
So all my build configurations build successfully and produce build artifacts on the Build Server #.BuildServer\system\artifacts{PROJECT}{several configurations}.. In my "Artifact Aggregating" configuration, I need to be able to reference what and where those artifacts are using variables that can be used in my ant/cmd script. i.e. I have Project A with configurations w, x, and y; how would I define/construct I variables of these configurations(w,x,y) that can be referenced by build configuration z. I looked at current Teamcity documentation i.e. http://www.jetbrains.net/confluence/display/TCD3/System+Properties+of+a+Build+Configuration#SystemPropertiesofaBuildConfiguration-ref; but I find this doesn't resolve my query.
Is there a way I can set up my artifact paths for configurations w, x and y to make the final task easier?
What would be the best way to accomplish this task? Any ideas are welcome.
This is how we do that.
Create n+1 Configuration(ZIP_ALL) and add dependency for all n projects see Dependency trigger
Create network share \\server\Build for aggregating project's building results
(you need cleanup strategy for that folder) - we simply drop all, our teammates create sub folders with SVN rev name (TC sets variable with revision value)
For each configuration create msbuild(or ant, or rake) script, that will build and zip (if you need) all output from build
Copy zip file or complete Output folder to common location (\\server\Build) see Copy Task
Create ant script for ZIP_ALL configuration that simply zips all files in common location
Publish that to TeamCity via Artifact Publishing