I am using AWS CodeBuild as a replacement for travis-ci.org. (Travis-CI was just too slow for our needs.) I would like to have CodeBuild test a variety of different configurations:
static vs shared libraries
gnu vs llvm compilers
debug vs release optimizations
I have a very simple buildspec.yml file that works well, but it doesn't seem to be able to generate a matrix of options like I can do on Travis-CI.
I could create a variety of buildspec.yml files with each one doing something a little differently, but that quickly gets out of hand with the different options that need to be considered.
What is the right/best way to do this with AWS CodeBuild?
I am from the AWS CodeBuild team. Thanks for your feedback and feature request. We'll pass this to our product management team to consider for future roadmap.
As you said, one option would be creating multiple buildspec.yml files and using overriding per build based on the desired build config.
Alternatively, you could have a single buildspec.yml that executes different build scripts based on an environment variable. For example you could have an environment variable called BUILD_CONFIG whose value will be read by your build script to determine which build configuration to use.
Environment variables can be overriden per build. Use the environmentVariablesOverride field in StartBuild:
http://docs.aws.amazon.com/codebuild/latest/APIReference/API_StartBuild.html#CodeBuild-StartBuild-request-environmentVariablesOverride
Related
I am using the AWS CLI task to deploy a Lambda layer. The build pipeline upstream looks like this:
It zips up the code, publishes the artifact and then downloads that artifact.
Now in the release pipeline I'm deploying that artifact via an AWS CLI command. The release pipeline looks like this:
I'm trying to figure out a way to dynamically get the current working directory so I don't need to hardcode it. In the options and parameters section you can see I'm trying to use $(Pipeline.Workspace) but it doesn't resolve correctly.
Is this possible?
Correct me if I am wrong, but I looks like you are running this in Azure Release? Not Pipelines?
If that is the case I think the variable you are looking for is $(Release.PrimaryArtifactSourceAlias) .
See the section of the document that talks about release specific variables: https://learn.microsoft.com/en-us/azure/devops/pipelines/release/variables?view=azure-devops&tabs=batch#default-variables---release
Yes. This is completely achievable.
From your screenshot, you are using the Release Pipeline to deploy the Artifacts.
In your situation, the $(Pipeline.Workspace) can only be used in Build Pipeline.
Refer to this doc: Classic release and artifacts variables
You can use the variable: $(System.ArtifactsDirectory) or $(System.DefaultWorkingDirectory)
The directory to which artifacts are downloaded during deployment of a release. The directory is cleared before every deployment if it requires artifacts to be downloaded to the agent. Same as Agent.ReleaseDirectory and System.DefaultWorkingDirectory.
The Problem
When using, cloudbuild.yaml files, specifically named for their build environment, such as cloudbuild-dev.yaml and cloudbuild-prod.yaml, and configured/targeted in the Trigger settings they aren't found/recognized when GCB reacts to a GitHub event (push etc).
However, it's working just fine when manually running the Trigger from GCB console.
When using an ordinarily named cloudbuild.yaml, in the root of the project, Cloud Build correctly runs the expected steps.
The Workaround
In short, there isn't an easy one (imo). But to get it run you need to use just a single _cloudbuild.yaml).
However, to effectively re-use that for both dev and prod environments one is blocked by this issue
I am building out a Sitecore farm with multiple Content Delivery servers. In the current process, I stand up the CD server and go through the manual steps of commenting out connection strings and enabling or disabling config files as detailed here per each virtual machine/CD server:
https://doc.sitecore.net/Sitecore%20Experience%20Platform/xDB%20configuration/Configure%20a%20content%20delivery%20server
But since I have multiple servers, is there any sort of global configuration file where I could dictate the settings I want (essentially a settings template for CD servers), or a tool where I could load my desired settings/template for which config files are enabled/disabled etc.? I have used the SIM tool for instance installation, but unsure if it offers the loading of a pre-determined "template" for a CD server.
It just seems in-efficient to have to stand up a server then config each one manually versus a more automated process (ex. akin to Sitecore Azure, but in this case I need to install the VMs on-prem).
There's nothing directly in Sitecore to achieve what you want. Depending on what tools you are using then there are some options to reach that goal though.
Visual Studio / Build Server
You can make use of SlowCheetah config transforms to configure non-web.config files such as ConnetionStrings and AppSettings. You will need a different build profiles for each environment you wish to create a build for and add the appropriate config transforms and overrides. SlowCheetah is available as a nuget package to add to your projects and also a Visual Studio plugin which provides additional tooling to help add the transforms.
Continuous Deployment
If you are using a continuous deployment tool like Octopus Deploy then you can substitute variables in files on a per environment and machine role basis (e.g. CM vs CD). You also have the ability to write custom PowerShell steps to modify/transform/delete files as required. Since this can also run on a machine role basis you can write a step to remove unnecessary connection strings (master, reporting, tracking.history) on CD environments as well as delete the other files specified in the Sitecore Configuration Guide.
Sitecore Config Overrides
Anything within the <sitecore> node in web.config can be modified and patch using Include File Patching Facilities built into Sitecore. If you have certain settings which need to be modified or deleted for a CD environment then you can create a CD-specific override, which I place in /website/App_Config/Include/z.ProjectName/WebCD and use a post-deployment PowrrShell script in Octopus deploy to delete this folder on CM environment. There are example of patches within the Include folder, such as SwitchToMaster.config. In theory you could write a patch file to remove all the config sections mentioned in the depoyment guide, but it would be easier to write a PowerShell step to delete these instead.
I tend to use all the above to aid in deploying to various environments for different server roles (CM vs CD).
Strongly recommend you take a look at Desired State Configuration which will do exactly what you're talking about. You need to set up the actual configuration at least once of course, but then it can be deployed to as many machines as you'd like. Changes to the config are automatically flowed to all machines built from the config, and any changes made directly to the machines (referred to as configuration drift) are automatically corrected. This can be combined with Azure, which now has capability to act as a "pull-server" through the Automation features.
There's a lot of reading to do to get up to speed with this feature-set but it will solve your problem.
This is not a Sitecore tool per se.
I've got an env.config in source control but pretty much the only things I can put in it are things that relate to all my various environments (production, staging). I've got environment specific settings that I want to add to the env.config file (for instance, the DB host) that will change from environment to environment. How can I handle these differences? Right now I'm doing it from the AWS console where I can manage it in the GUI on a per-environment basis, but I'd love to be able to change a lot of this stuff from git so I don't have to be logging into the console whenever I want to change something.
Is there any way to have multiple, environment specific config files?
So this has been posted before in the AWS forums. (https://forums.aws.amazon.com/thread.jspa?messageID=529373) So far there's only workarounds! The problem is that the .config files would require some logic to figure out what environment you're attempting to target. Personally I don't think any logic is required, as you could simply namespace the config settings based on the AWS environment name you're targeting.
I think your usecase is similar to what is discussed in How to configure Elastic Beanstalk for RDS
You may want to use 'eb branch'. You can then have multiple environments with different configurations.
More documentation on eb branch here
I have a list of Jenkins jobs that are independent, but it would be convenient if I could group them together to have them all run with the click of one button. Each of the projects is concerned with deployment, not compilation.
I've found the bulk-builder plugin, but to use bulk-builder it's necessary to specify a pattern each time you wish to invoke it.
I'm looking for a Jenkins plugin that will allow me to group projects together, and the Maven system seems to suggest this is possible: I'd make a top level build job that sets up dependencies on each of the jobs I wish to run, then I'd just need to run my top level job.
If possible, has anyone found Maven to be useful in managing dependencies of anything but Java? Would I be able to use it in the way I'm expecting?
EDIT: These are all .net projects
In jenkins, you can explicitly define other projects to build upon a successful build (search for downstream). This way, you can have one project trigger bunch of others effectively grouping them.
Maven is a great tool but I wouldn't use it for this purpose.