I am looking to add notifications to a build pipeline I am deploying in AWS via Terraform. I cannot seem to locate the resource which creates the status notifications in CodeBuild. Can someone let me know which resource this is?
You’ve not mentioned what sort of notification you are looking to create, so I won’t be able to provide some sample code, however, as per the AWS docs here, you can detect state changes jn CodePipeline using Cloudwatch events.
You can find the Terraform reference for CloudWatch Event Rules here, and you can follow the docs to create a resource that monitors CodePipeline for state changes using CloudWatch Events Rules.
Related
so im trying to run Terraform through CodePipeline. I need to manage a fleet of clusters. It seems CodePipeline is one of the good ways to trigger certain pipelines on some conditions.
I have a very simple requirement - i want to see the terraform execution in real time. i want to expose the CodePipeline run in a way that i can stream this. Is this where EventBridge is used. I tried to look at an EventBridge example here - https://medium.com/hackernoon/monitoring-ci-cd-pipelines-with-amazon-eventbridge-32177e2f2c3e - but it doesnt seem to be streaming run output in real time.
Which event or hook to should i attach to? And is CodePipeline even the right thing to use here ?
Which event or hook to should I attach to?
You're looking at the wrong AWS service. EventBridge is not for streaming log output. It is for discrete events, not a stream.
Your CodePipeline would be using a CodeBuild task to execute Terraform. Your CodeBuild task will be configured to log to AWS CloudWatch Logs. You can view the CloudWatch Logs output in the AWS CloudWatch web console, with the option to poll for new log output.
You can also do the same in a command line console with the aws logs tail command, documented here.
To do the same thing in your own code you would have to write your code to poll the CloudWatch Logs API in an loop.
And is CodePipeline even the right thing to use here?
Yes absolutely
Is there some way by which I can implement custom Advanced data events selector for cloud trail via terraform. Stuck on this for quite some time now. This is what I want to achieve via terraform.
Its not yet supported in terraform. You would have to set it up using local-exec yourself with aws-cli.
My requirement is to track any newly launch aws service/resources which does have 2 specific tags. What is the best way to do this? Right now thinking to have a lambda which will keep pinging any new resources is being launched or not which doesn't have specific tags.
Any other recommendation!
We do have a similar requirement where we want to ascertain that all of the newly created resources confirm certain requirements ( having specific tags and does not breach any security best practices).
We have implemented a soln which was based on AWS Config service
"AWS Config is a service that enables you to assess, audit, and evaluate the configurations of your AWS resources. Config continuously monitors and records your AWS resource configurations and allows you to automate the evaluation of recorded configurations against desired configurations. With Config, you can review changes in configurations and relationships between AWS resources, dive into detailed resource configuration histories, and determine your overall compliance against the configurations specified in your internal guidelines. This enables you to simplify compliance auditing, security analysis, change management, and operational troubleshooting." Reference: https://aws.amazon.com/config/
Consolidated list of the resources supported by AWS Config can be found at: https://docs.aws.amazon.com/config/latest/developerguide/resource-config-reference.html
The High-level design to achieve this as below:
The following needs to be done to implement the architecture
Enable Conf service in the region which you want to observe.
Enable config change recording from config console.
Create a lambda that evaluates the json event received for the config changes on AWS resources monitored by Config service and check for tags (in your case).
Create a custom config rule which observes the event for resources of your interest delegate those events to the lambda which you have created in step 3.
Lambda can send events to SNS for noncompliant resources. From SNS you can trigger an email
We have two Jenkins pipeline ; one called log monitoring and another as alert trigger. Both of these pipelines get triggered on any changes to the terraform scripts in Bitbucket.
The pipeline works fine, and the AWS resources are getting created successfully.
The problem that we are facing here is :
The log monitoring pipeline creates an AWS resource, whose ARN we want to fetch from the AWS console and use in the alert trigger pipeline.
Any thoughts how we can achieve this as we want to automate all the pipeline instead of manually fetching the ARN and triggering the downstream pipeline.
You can attach tags to your existing resources and use those to retrieve their respective ARNs using AWS CLI in shell scripts. From that output you can further use that ARN value inside your pipeline in a dynamic manner.
Notifications: https://docs.aws.amazon.com/codecommit/latest/userguide/how-to-repository-email.html
Triggers: https://docs.aws.amazon.com/codecommit/latest/userguide/how-to-notify.html
The official document states that for CodeCommit repository 'events which follow CloudWatch Event Rules' (like pull requests), we use Repository Notifications.
Whereas for CodeCommit repository events which are just 'operational events' (like creating branches, pushing code to a branch), we use Repository Triggers.
I don't understand the difference between 'events which follow CloudWatch Event Rules' and 'operational events'. For me, both pull requests and pushing code to branch seem similar events.
Thus, confused between why we need both Repository Notifications and Repository Triggers.
I have asked the same question today and I found this on docs:
Repository notifications are different from repository triggers. Although you can configure a trigger to use Amazon SNS to send emails about some repository events, those events are limited to operational events, such as creating branches and pushing code to a branch. Triggers do not use CloudWatch Events rules to evaluate repository events. They are more limited in scope. For more information about using triggers, see Manage Triggers for a Repository.
IMO, AWS documentation has not clearly stated the difference between notification and triggers and cloudwatch events. Here is my understanding :
Notifications should be used for literal notification and not for taking action based on them.
Triggers are supposed to initiate action. So, if I need to invoke some service based on this event on which trigger is based, I would do that and hence the option to integrate Lambda service. In a way to add automation after codecommit events.
However, Cloudwatch Events provide a wide variety of integration option for codecommit events which are not available with trigger.