Is there anyway to Query AWS Service Status e.g. if Kinesis Stream is Up or not, or S3 Storage Service is available or not ? And on the basis of status, I want to execute certain CURL commands ?
I am looking to get a status for below services and then use it in our custom StatusPage
S3
API Gwy
Kinesis
SQS
Route53 + Cloudfront
I know there is statuspage for AWS Services but not sure how to use it programmatically and publish a CURL command.
The Service Health Dashboard is usually delayed and not very useful for real-time monitoring of AWS health status.
Instead, you could use AWS Health from your account, which provides much more up-to-date events. What's more you can setup automated notifications and response to any issues detected as described in:
Monitoring AWS Health events with Amazon CloudWatch Events
Related
We have enabled VPC Flow logs for our ALL VPC's, now I am searching for creating Alert to detect the Malicious activity and send mail when it detects.
What is the process to achieve this?
Use CloudWatch and Amazon SNS notifications.
Another good service is AWS Config, which will check your configuration and its changes.
See: Publish flow logs to CloudWatch Logs - Amazon Virtual Private Cloud
Im looking for lambda that can trigger Gitlab-ci pipeline to deploy specific branches and send results to slack.
Thx.
Trigger a pipeline
As per GitLab Trigger API manual:
To trigger a job you need to send a POST request to GitLab’s API endpoint:
curl -X POST <API url>/projects/<your_awesome_gitlab_project>/trigger/pipeline
The required parameters are the trigger’s token and the Git ref on which the trigger will be performed. Valid refs are the branch and the tag. The :id of a project can be found by querying the API or by visiting the CI/CD settings page which provides self-explanatory examples.
Watching a pipeline
To check pipeline results, use CloudWatch Events:
You can set up a rule to run an AWS Lambda function on a schedule. This tutorial shows how to use the AWS Management Console or the AWS CLI to create the rule. If you would like to use the AWS CLI but have not installed it, see the AWS Command Line Interface User Guide.
To check jobs status, use: Get a single pipeline or List project pipelines API calls.
curl --header "PRIVATE-TOKEN: " "https://gitlab.example.com/api/v4/projects/1/pipelines/46"
Inform on Slack
To send Slack notifications with lambda, use this tutorial:
Creating an AWS Lambda Function and API Endpoint | Slack
Two cents about endpoint security
CI Trigger is secured by token. In general, it's enough for securing your endpoints.
But, if the approach isn't enough, there are some techniques to "hide" endpoints:
client IP whitelisting with GitLab
AWS Security Groups for Lambda or for EC2
Securing URLs with Nginx or with HAProxy
Suppose multiple aws services are running (like EC2,S3) and If someone changes the configuration of these services , I want to know immediately and want to pull the updated configuration. So how can i listen to these services to pull configuration immediately . I went through couple of options like aws SNS and all. But i don't want to implement this using any aws services for this.
I am using spring boot application. Any help is appreciated , Thanks in advance.
The two services you should be interested in are:
AWS CloudTrail:
AWS CloudTrail is an AWS service that helps you enable governance, compliance, and operational and risk auditing of your AWS account. Actions taken by a user, role, or an AWS service are recorded as events in CloudTrail. Events include actions taken in the AWS Management Console, AWS Command Line Interface, and AWS SDKs and APIs.
AWS Config:
AWS Config provides a detailed view of the configuration of AWS resources in your AWS account. This includes how the resources are related to one another and how they were configured in the past so that you can see how the configurations and relationships change over time.
You can also configure Amazon CloudWatch Events rules that trigger when a particular API call is made (eg when a user changes some configuration). This could trigger a notification, a Lambda function, etc.
See:
Creating a CloudWatch Events Rule That Triggers on an AWS API Call Using AWS CloudTrail - Amazon CloudWatch Events
How to monitor AWS account activity with Cloudtrail, Cloudwatch Events and Serverless
How I can retrieve the AWS services statuses programmatically? Is there any API which will provide me the status like below URL, which can give me health status of desired services which my cloud formation stack will use.
https://status.aws.amazon.com/
You can get the status of the services that you are using.
For API access, see: AWS Health API
The AWS Health API provides programmatic access to the AWS Health
information that is presented in the AWS Personal Health Dashboard.
You can get information about events that affect your AWS resources
I have a java web application running on Tomcat deployed on an EC2 instance. Is there any way I can monitor/set alarms for when the web application goes down or stops responding? Essentially what I would like to do is to check if a HTTP request to the web app responds with status 200. If it does not respond with 200 (for a few times) then it should raise an alarm and send an e-mail to some ops people.
I know there are third party options like Nagois / uptimerobot that I could use but I wanted to know if there are any AWS offerings for this? Is it possible to set up such automated monitoring using AWS Cloud Watch? I could not find a way to do this based on what I read up about Cloud Watch. If this isn't the sort of thing Cloud Watch can handle, then is there another AWS service suited for this?
I think Port Monitoring Feature is available under AWS Beanstalk.
You can consider checking this http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.healthstatus.html
Ashutosh,
Ec2 is an IAAS service from AWS and you will not have an AWS offering to monitor your Tomcat server. However, you have custom-built solutions, which I think you are not looking for here.
However, if you are using an Application Load balancer or Beanstalk you get options to trigger alarms.
Yes , you can achieve it through a cloudwatch . collect your logs with a cloudwatch agent and upload it on cloudwatch logstream. below is the reference url for configuring cloudwatch agent.
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html
After that with "create matrix filter" you can set up an email trigger as per your requirements.
https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/MonitoringPolicyE
https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Counting404Responses.htmlxamples.html