I would like to know how I can create a filtering policy for AWS SNS subscription that would check a message attribute value but only if that attribute is present. By default if I check an attribute value but attribute is not present message is ignored e.g:
"customer_interests": ["paintball"]
I also found this for attribute presence checking:
"customer_interests": [{"exists": true}]
But I'm not sure how to combine this two checks into a single policy.
I've tried the obvious thing:
{
"customer_interests": [{"exists": false}, "paintball"]
}
but it doesn't work.
i have tried the following thing ( your obvious thing ) and it worked for me, i didn't have to even wait for 15 minutes (clearly eventual consistency is not the issue )
Used the following policy
{
"customer_interests": [
{
"exists": false
},
"paintball"
]
}
Created a sns topic with email-based subscription
added filter policy to subscriber
tried with three use case
Case 1
message hello
attribute type string
atrribute name customer_interests
atrribute value paintball
status recieved
Case 3
message hello
attribute type string
atrribute name customer_interests
atrribute value xyz
status not recieved
Case 3
message hello
attribute type string
atrribute name jatin
atrribute value test
status recieved
You apply OR logic in SNS subscription filter policies, by assigning multiple values to an attribute name.
Your example:
{
"customer_interests": [{"exists": false}, "paintball"]
}
should result in filtering messages that:
do not have the customer_interests message attribute
OR have the customer_interests message attribute set to paintball
Please note that additions or changes to a subscription filter policy require up to 15 minutes to fully take effect, as Amazon SNS is eventually consistent.
Changes will not always take effect immediately.
Related
Here is a modified version of an Event type I am receiving in my handler for a lambda function with a DynamoDB someTableName table trigger that I logged using cargo lambda.
Event {
records: [
EventRecord {
change: StreamRecord {
approximate_creation_date_time: ___
keys: {"id": String("___")},
new_image: {
....
"valid": Boolean(true),
},
...
},
...
event_name: "INSERT",
event_source: Some("aws:dynamodb"),
table_name: None
}
]
}
Goal: Correctly filter with event_name=INSERT && valid=false
I have tried a number of options, for example;
{"eventName": ["INSERT"]}
While the filter is added correctly, it does not trigger the lambda on item inserted.
Q1) What am I doing incorrectly here?
Q2) Why is table_name returning None? The lambda function is created with a specific table name as trigger. The returned fields are returning an option (Some(_)) so I'm asssuming it returns None if the table name is specified on lambda creation, but seems odd to me?
Q3) From AWS Management Console > Lambda > ... > Trigger Detail, I see the following (which is slightly different from my code mentioned above), where does "key" come from and what does it represent in the original Event?
Filters must follow the documented syntax for filtering in the Event Source Mapping between Lambda and DynamoDB Streams.
If you are entering the filter in the Lambda console:
{ "eventName": ["INSERT"], "dynamodb": { "NewImage": {"valid": { "BOOL" : [false]}} } }
The attribute name is actually eventName, so your filter should look like this:
{"eventName": ["INSERT"]}
I have two kinds of messages in the AWS CloudWatch and would like to select the first field that has some text in it. For example
Mesasge 1:
"message": {
"message": "I am the first priority"
}
Message 2:
"message": {
"err": {
"message": "I am second priority"
}
}
I would like to have these in a single column of the CloudWatch table depending which one is present. Is there any way to do this? Something like this (which obviously doesn't work):
fields #timestamp, ispresent(message.message) ? message.message : message.err.message
Apparently coalesce function is what I needed. It selected the first value that is not null:
fields #timestamp, component, coalesce(message.message, message.err.message) as TheMessage
More info at CloudWatch Logs Insights query syntax
We have two types of SNS messages coming in:
1. has MessageAttributes empty like this:
"MessageAttributes": {}
2. has MessageAttributes coming in like this:
"MessageAttributes": {
"Generator": {
"Type": "String",
"Value": "some-service"
}
}
I would like to use a filter subscription policy that ignores the second type but passes the first type to the subscriber.
So I tried this for the policy:
{
"Generator": [
{
"exists": false
}
]
}
I thought this would mean it will only pass along messages that do NOT contain the Generator key in MessageAttributes
However I am seeing now that no messages are getting passed along.
The AWS Subscription Filter docs seem to support this as a solution, but they only show the opposite way of checking that a key does exist, so I'm not sure if they support checking a key doesn't exist: https://docs.aws.amazon.com/sns/latest/dg/sns-subscription-filter-policies.html#attribute-key-matching
Is this possible?
The answer from #David Adams is out of date. See the Attribute key matching docs.
Use "exists": false to return incoming messages that don't include the specified attribute.
It is now possible to exclude any messages that have a particular key by using the policy:
{
"key": [
{
"exists": false
}
]
}
Late response but may be helpful to someone.
Filtering out by lack of existance is not possible. See the bottom of https://docs.aws.amazon.com/sns/latest/dg/sns-subscription-filter-policies.html#attribute-key-matching
Note: You cannot use the exists operator to match messages in which an attribute does not exist. Filtering will NOT match any messages if you set [{"exists": false}].
You could pass a string 'null' or similar to the generator attribute if it is non existant maybe?
I have customised the managed rule - required-tags and modified the lambda function to extend upto 9 tags and more.
The code seems to work and gives me the expected result.
This rule checks for tags given in "Rule Parameters". Empty and partially tagged are non compliant; fully tagged are compliant.
The problem I face is when a new EC2 instance is created and the custom rule triggers with configuration changes gives me "no tags" even when "name" tag is present.
When I re-evaluate for the second time, the expected result is got (missing tags except name tag)
The condition goes like this
if evaluation["compliance_type"] == "NON_COMPLIANT":
print ("NON_COMPLIANT")
if len(evaluation["current_tags"]) > 0:
print ("Non zero tags")
// evaluation report
else:
print ("Zero tags")
// evaluation report
EC2 instance was laucnhed around 12:39 PM
Cloudwatch logs after automatic trigger(configuration changes) around 12:41 PM
{
'current_tags': [],
'compliance_type': 'NON_COMPLIANT',
'annotation': 'Name, Customer, Environment, etc not present'
}
Cloudwatch logs after manual re-evaluation around 12:43 PM
{
'current_tags': [{u'value': u'instance_name', u'key': u'Name'}],
'compliance_type': 'NON_COMPLIANT',
'annotation': 'Customer, Environment, etc not present'
}
The event payload passed to lambda function (trigger: configuration changes) at instance creation time has current_tags: empty (even though name tag is added during instance creation; exists). Is there a way to find out how and when the tags gets added to the instance. Or the trigger can be delayed (not periodic trigger)
I would like to send an SMS message from an AWS Lambda function using the boto3 publish method to notify the user of issues via SMS. My lambda function is written in Python and I am using the boto3 module. My lambda function has full rights to SNS. I have this code,
sns = boto3.client('sns')
sns.publish(
PhoneNumber = '+11234567890',
Message = 'Simple text message'
)
According to the boto3 documentation, the publish method accepts the following parameters,
response = client.publish(
TopicArn='string',
TargetArn='string',
PhoneNumber='string',
Message='string',
Subject='string',
MessageStructure='string',
MessageAttributes={
'string': {
'DataType': 'string',
'StringValue': 'string',
'BinaryValue': b'bytes'
}
}
)
It requires a "Message" parameter and one of the following three parameters as described in the docs:
TopicArn (string) -- The topic you want to publish to.
If you don't specify a value for the TopicArn parameter, you must
specify a value for the PhoneNumber or TargetArn parameters.
TargetArn (string) -- Either TopicArn or EndpointArn, but not both.
If you don't specify a value for the TargetArn parameter, you must
specify a value for the PhoneNumber or TopicArn parameters.
PhoneNumber (string) -- The phone number to which you want to deliver
an SMS message. Use E.164 format.
If you don't specify a value for the PhoneNumber parameter, you must
specify a value for the TargetArn or TopicArn parameters.
When my code is executed a parameter validation error is returned. It states,
Unknown parameter in input: "PhoneNumber", must be one of: TopicArn,
TargetArn, >Message, Subject, MessageStructure, MessageAttributes".
So the documentation seems to indicate that PhoneNumber is a valid parameter, but when used, an error occurs and the feedback from the error indicates that PhoneNumber is not a possible parameter. I suspect I am missing something obvious and simple, but could use some help.
I know there are other avenues to send SMS messages such as email gateways and other vendor supplied solutions like Twilio, but I would like to pursue the SNS based route and understand where I have gone wrong.
Actually your example looks right. Here is what I tried
import boto3
sns = boto3.client('sns')
number = '+17702233322'
sns.publish(PhoneNumber = number, Message='example text message' )
Worked like a charm. I recommend using awscli configured with your root account credentials first and take the code for a test drive. Once its working either create a new user with just the rights you need, or apply it to an Instance role.
You need to create a policy that allows SNS:Publish on resource:* (allow texting to everyone) or resource: '+17702233322' (allow text to a specific number).