Cloudwatch rule that trigger SNS in case pattern do NOT match completely - amazon-web-services

Is it possible to create CloudWatch Rule that triggers SNS, when pattern do NOT match completely?
With following example I hope that question will be more clear:
{
"source": [
"aws.ec2"
],
"detail": {
"eventSource": [
"ec2.amazonaws.com"
],
"eventName": [
"RunInstances"
]
}
}
Additionally I want to specify region: "awsRegion": "eu-central-1" but (here is tricky part) want SNS to be triggered when awsRegion is NOT eu-central-1.
Idea is to receive a notification when someone makes a mistake and runs an instance in the wrong region.
Also will add more rules once I know how to do, so the question is not exactly for the region, but general.
Thanks in advance!
TeoVal

It's currently not possible to trigger when a pattern does NOT match.
You will have to receive notifications for all regions on your SNS topic and implement your own logic with a Lambda function subscribed to your topic (or directly with Lambda as a target).
Also note that it's possible to restrict users from making API calls in specific region using a policy similar to the following (restricts call to us-east-1 and eu-central-1 regions):
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "RegionsRestriction",
"Effect": "Deny",
"Action": "*",
"Resource": "*",
"Condition": {
"StringNotEquals": {
"aws:RequestedRegion": [
"us-east-1",
"eu-central-1"
]
}
}
}
]
}

Related

Jobs from specific AWS Batch permissions

How to allow only jobs from a certain AWS Batch queue (and based on a specific job description) to publish to the specific SNS topic?
I though about attaching to jobs IAM policy with the statement:
{
"Effect": "Allow",
"Action": "sns:Publish",
"Resource": ["<arn of the specific SNS topic"]
"Condition": {"ArnEquals": {"aws:SourceArn": "arn:aws:???"}}
}
But what should be the source ARN? ARN of the job queue, ARN of the job definition? Or maybe this should be set up completely differently?
I had a similar experience when worked with AWS Batch jobs executed in Fargate containers which follow the same principles as ECS in scope of assigning roles and permissions.
If you are going to publish messages into specific topic from the code executed inside of your container, then you should create a role with necessary permissions and then use its ARN in the JobRoleArn property of your job definition.
For example (there can be minor mistakes in the code below, but I am just trying to explain the concept here):
Role cloudformation:
"roleresourceID": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Principal": {
"AWS": "*"
}
}
],
"Version": "2012-10-17"
},
"RoleName": "your-job-role"
}
}
Policy attached to the role:
"policyresourceid": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyDocument": {
"Statement": [
{
"Action": "sns:Publish",
"Effect": "Allow",
"Resource": "<arn of the specific SNS topic>"
}
],
"Version": "2012-10-17"
},
"PolicyName": "your-job-role-policy",
"Roles": [
{
"Ref": "roleresourceID"
}
]
}
}
And finally attach role to the Job Definition:
....other job definition properties
"JobRoleArn": {
"Fn::GetAtt": [
"roleresourceID",
"Arn"
]
}
Of course you may structure and format roles and policies in way you like, the main idea of this explanation is that you need to attach proper role using JobRoleArn property of your job definition.

S3 Create event notifcation to SQS "Unable to validate the following destionation configurations"

Using strictly the management console I created a new S3 bucket and SQS with the default settings. I am then trying to create a notification to SQS queue but whenever I try to create it, it fails with "Unable to validate the following destination configurations". After a couple of hours of searching this error seems mostly related to creating the resources in the wrong order or something with Cloudformation, but I am not using that. Also most of the docs describe connecting the S3 event to an SNS topic, not SQS. I have also spent a lot on time trying to figure out if its policy related. For my SQS policy I have added the following:
{
"Version": "2012-10-17",
"Id": "__default_policy_ID",
"Statement": [
{
"Sid": "__default_statement_ID",
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": "SQS:SendMessage",
"Resource": "arn:aws:sqs:eu-central-1:123:my-sqs",
"Condition": {
"StringEquals": {
"aws:SourceAccount": "123"
},
"ArnLike": {
"aws:SourceArn": "arn:aws:s3:::my-bucket"
}
}
}
]
}
Which I think is correct. But this did not help.
How can I get rid of this error and get it working?

Access control of AWS SQS subscription for AWS SNS topic

I have a NodeJS application that publishes message to AWS SNS topic string and a AWS SQS subscription for the same. On the SQS console, I can see the published message. However, I am not clear with the access policy of the SQS queue.
This answer mentions the use of "Principal": "*" - but, that is very broad. One could probably use "Principal" : {"AWS": "*"}; but, that isn't narrow either.
{
"Version": "2012-10-17",
"Id": "Policy1607949016538",
"Statement": [
{
"Sid": "Stmt1607949012567",
"Effect": "Allow",
"Principal": "*",
"Action": [
"sqs:ReceiveMessage",
"sqs:SendMessage"
],
"Resource": "arn:aws:sqs:ap-south-1:463113000000:orders"
}
]
}
Questions
While delivering a message to SQS queue, as a result of subscription, which user is in effect? Same as the one who published to the topic?
I could get the messages to flow into the queue only when I used "Principal" : {"AWS": "*"}. So, how should I define a restrictive policy such that messages are written to queues only as a result of subscription?
What is the equivalent in the AWS SQS CLI to create a queue with "Principal" : {"AWS": "*"} permissions?
The only user that matters is the one that qualifies for the policy as defined for subscription and SQS access policy.
The Condition in policy document can make the overall policy restrictive. See example below.
Adding SQS Permissions with conditions using AWS CLI Command
Example policy document restricting access to account ID.
{
"Version": "2012-10-17",
"Id": "Policy1607960702002",
"Statement": [
{
"Sid": "Stmt1607960701004",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"sqs:ReceiveMessage",
"sqs:SendMessage"
],
"Resource": "arn:aws:sqs:ap-south-1:463113000000:orders",
"Condition": {
"StringEquals": {
"aws:SourceAccount": "463113000000"
}
}
}
]
}

How to send SMS through SNS and Cloudwatch?

I am trying to send SMS to my Mobile when my EC2 instance stops.
I am automatically stopping my EC2 instance and now I want to send SMS to my mobile when it stops.
I created SNS topic with my mobile no. as subscriber.
I created an Alarm when the EC2 stops.
Under SNS > Mobile > Text messaging (SMS) > Text messaging preferences (Edit):
a. I selected "Default message type" as "Transactional".
b. I created a new IAM role.
IAM role policy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents",
"logs:PutMetricFilter",
"logs:PutRetentionPolicy"
],
"Resource": [
"*"
]
}
]
}
SNS topic access policy
{
"Version": "2008-10-17",
"Id": "__default_policy_ID",
"Statement": [
{
"Sid": "__default_statement_ID",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"SNS:Publish",
"SNS:RemovePermission",
"SNS:SetTopicAttributes",
"SNS:DeleteTopic",
"SNS:ListSubscriptionsByTopic",
"SNS:GetTopicAttributes",
"SNS:Receive",
"SNS:AddPermission",
"SNS:Subscribe"
],
"Resource": "arn:aws:sns:us-west-2:account-id:sns-topic-name",
"Condition": {
"StringEquals": {
"AWS:SourceOwner": "account-id"
},
"ArnLike": {
"AWS:SourceArn": "arn:aws:cloudwatch:us-west-2:account-id:alarm:*"
}
}
}
]
}
When the alarm is triggered, I am getting the below error:
{
"actionState": "Failed",
"stateUpdateTimestamp": 1561102479560,
"notificationResource": "arn:aws:sns:us-west-2:account-id:sns-topic-name",
"publishedMessage": null,
"error": "Resource: arn:aws:cloudwatch:us-west-2:account-id:alarm:alarm-name is not authorized to perform: SNS:Publish on resource: arn:aws:sns:us-west-2:account-id:sns-topic-name"
}
I am unable to understand what permission is it expecting.
The cause of the error is most likely due to the policy having incorrect values. I'm not sure which values you changed to protect sensitive values, but you'd need to update sns-topic-name and account-id.
However, I would recommend another way of achieving your goals...
You can use Amazon CloudWatch Events to look out for a specific event (eg an instance changing state to Stopped) and have it send a message to Amazon SNS directly (without using an Alarm).
The steps are:
In the Amazon CloudWatch console, click Rules
Create rule
Service name: EC2
Event type: EC2 Instance State-change Notification
Specific state(s): Stopped
Choose Any instance or Specific instance Id(s)
On the right, under Targets, click Add target
SNS topic
Select your topic
This will then send a message whenever the instance stops.
It seems the error is due to missing permissions on your IAM role for publishing messages to an SNS topic. Make arrangements to attach necessary permissions to the role you use or to the user, like this:
{
"Id": "Policy1415489375392",
"Statement": [
{
"Sid": "AWSConfigSNSPolicy20150201",
"Action": [
"SNS:Publish"
],
"Effect": "Allow",
"Resource": "arn:aws:sns:region:account-id:myTopic",
"Principal": {
"AWS": [
"account-id1",
"account-id2",
"account-id3",
]
}
}
]
}

Unable to trigger AWS Lambda by upload to AWS S3

I am trying to build a Kibana dashboard fed with twitter data collected via AWS Kinesis firehose where data passes into an S3 bucket which triggers a Lambda function which passes the data to AWS Elastic Search and then to Kibana. I am following this blog https://aws.amazon.com/blogs/big-data/building-a-near-real-time-discovery-platform-with-aws/
The data is loading into the S3 bucket correctly but it never arrives in Kibana, I believe this is because the Lambda function is not being triggered by events in S3 as I would have hoped (there are no invocations or logs). I think this is because I have not set permissions properly. The Lambda function can be invoked manually by the test event.
On the Lambda function page I chose an existing role which I called lambda_s3_exec_role which has the AWSLambdaExecute policy attached to it but I feel I'm missing something else more specific to S3. I have been unable to follow this line in the blog in the create lambda function section because I do not recognise those options:
"10. Choose lambda_s3_exec_role (if this value does not exist, choose Create new role S3 execution role)."
Can anyone help me create the appropriate role/policy for the Lambda function, or spot what the problem may be?
From view permissions on the Lambda function I currently have:
FUNCTION POLICY
{
"Version": "2012-10-17",
"Id": "default",
"Statement": [
{
"Sid": "****",
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "****",
"Condition": {
"ArnLike": {
"AWS:SourceArn": "arn:aws:s3:::****"
}
}
}
]
}
EXECUTION ROLE
{
"roleName": "lambda_s3_exec_role",
"policies": [
{
"document": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:*"
],
"Resource": "arn:aws:logs:*:*:*"
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::*"
}
]
},
"name": "AWSLambdaExecute",
"id": "****",
"type": "managed",
"arn": "arn:aws:iam::aws:policy/AWSLambdaExecute"
}
]
}
The permissions you have listed look OK so I am going to try provide some steps that might help find the issue as it is difficult to understand specifically where your issue might be.
Does the execution role have the trust relationship with a trusted entity of lambda.amazonaws.com
Does your event prefix match the prefix in firehose. In the tutorial they are both twitter/raw-data/. If firehose is writing to a path that isn't the event prefix then the event won't be invoked.
Does the lambda trigger any errors when you manually invoke it
Does the lambda write to the logs when you manually invoke it
Test the lambda using dummy data (example data below)
CLI
aws lambda invoke \
--invocation-type RequestResponse \
--function-name helloworld \
--region region \
--log-type Tail \
--payload file://dummy_event.json \
--profile adminuser \
outputfile.txt
Example data
source
dummy_event.json
{
"Records":[
{
"eventVersion":"2.0",
"eventSource":"aws:s3",
"awsRegion":"us-west-2",
"eventTime":"1970-01-01T00:00:00.000Z",
"eventName":"ObjectCreated:Put",
"userIdentity":{
"principalId":"AIDAJDPLRKLG7UEXAMPLE"
},
"requestParameters":{
"sourceIPAddress":"127.0.0.1"
},
"responseElements":{
"x-amz-request-id":"C3D13FE58DE4C810",
"x-amz-id-2":"FMyUVURIY8/IgAtTv8xRjskZQpcIZ9KG4V5Wp6S7S/JRWeUWerMUE5JgHvANOjpD"
},
"s3":{
"s3SchemaVersion":"1.0",
"configurationId":"testConfigRule",
"bucket":{
"name":"sourcebucket",
"ownerIdentity":{
"principalId":"A3NL1KOZZKExample"
},
"arn":"arn:aws:s3:::sourcebucket"
},
"object":{
"key":"HappyFace.jpg",
"size":1024,
"eTag":"d41d8cd98f00b204e9800998ecf8427e",
"versionId":"096fKKXTRTtl3on89fVO.nfljtsv6qko"
}
}
}
]
}
Struggled with this for a long time and eventually realized that your rule that triggers the lambda cannot have exactly the same name as the lambda itself or it won't work.