How to have multiple codepipeline triggers in aws? - amazon-web-services

Question

How can I have Aws CodePipeline be triggered by multiple sources? Imagine I want to have the same pipeline to be triggered whenever I push to two different repositories? Plus, the build stage must know which repository triggered the pipeline and pull from the right repository

Answer 1

Well, it will depend on the pipeline itself. Aws says codepipelines are made per project only. However, one way you could tackle this problem is by:
building a lambda function what triggers codebuild
the lambda function will have as many triggers as the number of repositories you want to trigger the same pipeline
the lambda function will pass environment variables to CodeBuild and trigger its execution
CodeBuild will work out which repo to pull from depending on the value of the environment variable
How-To:
To begin with log into the aws console and head to Lambda functions
Create a new function - or edit one, if you prefer
Choose to create a new function from scratch if you created the function on the step above
Chose the running environment --- which in this example is going to be NodeJs 10.x but you can choose the one of your preference
Use default permission settings or use an already created role for this function if you have them already. This is important because we will be editing these permissions later
Click create function to create it. - If you already have a function you can jump to the next step!
You should be prompted with this screen. Click on "Add trigger"
After that you must choose your provider. This could be any of the options, including CodeCommit. However, if you want another service to be your trigger and it is not listed here you can always create an SNS topic and make that service subscribe to it and then make the SNS topic be the trigger for your function. This is going to be the subject of another tutorial later on...
In this case it is going to be CodeCommit and you should choose it from the list. Then you will be prompted with a screen like this one to configure your preferences
One thing to keep in mind is that the Event name is quite crucial since we are going to use it in our function to choose what is going to happen. Thus, choose it properly
After that you should end up with something like this Now it's time to code our function
Since we are going to have multiple repositories trigger the same CodeBuild project you can always refer to the AWS SDK CodeBuild documentation here
The method we are going to use is the CodeBuild StartBuild method
To configure it properly we are going to have to know in which region our build project is. You can see it by going to your project on the aws console and looking at the url prefix here
Coming back to our lambda function we are going to create a json file that will store all of our data that is going to be trasfered to codebuild when the function runs. It is important to name it WITH THE EXTENSION The value of the environment variables are going to depend from trigger to trigger and it is now that the trigger name is so important. It is going to be the key of our selector. The selector is going to take the event trigger name and use it to look into the json and define the environment variables
In the repositories.json file we are going to put all the data we want and, if you know a little bit of json you can store whatever you want and pass it to our function when you want it to be passed to the CodeBuild as an environment variable
The code is as follows:
const AWS = require('aws-sdk'); // importing aws sdk
const fs = require('fs'); // importing fs to read our json file
const handler = (event, context) => {
AWS.config.update({region:'us-east-2'}); // your region can vary from mine
var codebuild = new AWS.CodeBuild(); // creating codebuild instance
//this is just so you can see aws sdk loaded properly so we are going to have it print its credentials. This is not required but shows us that the sdk loaded correctly
AWS.config.getCredentials(function(err) {
if (err) console.log(err.stack);
// credentials not loaded
else {
console.log("Access key:", AWS.config.credentials.accessKeyId);
console.log("Secret access key:", AWS.config.credentials.secretAccessKey);
}
});
var repositories = JSON.parse(fs.readFileSync('repositories.json').toString());
var selectedRepo = event.Records[0].eventTriggerName;
var params = {
projectName: 'lib-patcher-build', /* required */
artifactsOverride: {
type: 'CODEPIPELINE', /* required*/
},
environmentVariablesOverride: [
{
name: 'name-of-the-environment-variable', /* required */
value: 'its-value', /* required */
type: 'PLAINTEXT'
},
{
name: 'repo-url', /* required */
value: 'repositories[selectedRepo].url', /* required */
type: 'PLAINTEXT'
}
/* more items */
],
};
codebuild.startBuild(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
});
};
exports.handler = handler;
Then, in our build project we can perform a test example to see if our function worked by trying to print the environment variable we passed from our json file into our function and out to the CodeBuild Project
For our function to be able of starting the build of our CodeBuild project we must grant access to it. So:
Go back to your lambda and click on the permissions tab
Click on Manage these permissions
Click your policy name but keep in mind it will NOT be the same as mine
Edit your function's policy
Click to add permissions
Click to choose a service and choose CodeBuild
Add the permission for our function to start the build
Now you have two choices: You can either use this permission on all build projects or restrict it to specific projects. In our case we are going to restrict it since we have only one build project
To restrict you have to click the dropdown button on resources, choose specific and click on Add ARN
Then we need to open a new tab and go to our CodeBuild project. Then, copy the project's ARN
Paste the ARN on the permissions tab you previously were on and click on save changes
Review the policy
Click to save the changes
Now, if you push anything to the repository you configured the build project should get triggered and, in the build logs session you should see the value of the variable you set in your function