I want a certain HTTPS service to be called every time a file has been uploaded to an S3 bucket.
I have created the S3 bucket and a SNS topic with a verified subscription with the HTTPS service as an endpoint.
I can publish a message on the SNS topic via the AWS UI, and see that the HTTPS service is called as expected.
On the S3 bucket I created an Event, which should link the bucket and the topic. On my first attempt I got an error because the bucket was not allowed to write to the topic, so c.f. the documentation, I changed the topic access policy to:
{
"Version": "2008-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "sns:Publish",
"Resource": "arn:aws:sns:eu-central-1:TOPIC_ID:OrderUpdates",
"Condition": {
"StringEquals": {
"AWS:SourceAccount": "ACCOUNT_ID"
},
"ArnLike": {
"AWS:SourceArn": "arn:aws:s3:*:*:*"
}
}
}
]
}
where TOPIC_ID is the topic owner id which can be seen when the topic is shown in the AWS UI, and the ACCOUNT_ID is the account id shown under account settings in the AWS UI.
This change in the topic access policy allowed me to create the event on the bucket:
When I call the API method getBucketNotificationConfiguration I get:
{
"TopicConfigurations": [
{
"Id": "OrderFulfilled",
"TopicArn": "arn:aws:sns:eu-central-1:TOPIC_ID:OrderUpdates",
"Events": [
"s3:ObjectCreated:*"
]
}
],
"QueueConfigurations": [],
"LambdaFunctionConfigurations": []
}
But the HTTPS service is not called. What am I missing in this setup, that will trigger the HTTPS service to be called by the SNS topic subscription every time a file is uploaded to the S3 bucket?
Thanks,
-Louise
Having the same issue S3 upload event does not trigger sns message even though our sns access policy is correctly set. Turns out we can NOT use the Enable encryption option, since S3 events are triggered via CloudWatch Alarms which do not work with SNS encrypted topics as of now.
Switch back to Disable encryption option, everything works again.
To reproduce this situation, I did the following:
Created an Amazon SNS topic and subscribed my phone via SMS (a good way to debug subscriptions!)
Created an Amazon S3 bucket with an Event pointing to the Amazon SNS topic
I received this error message:
Unable to validate the following destination configurations. Permissions on the destination topic do not allow S3 to publish notifications from this bucket.
I then added the policy you show above (adjusted for my account and SNS ARN)
This allowed the Event to successfully save
Testing
I then tested the event by uploading a file to the S3 bucket.
I received an SMS very quickly
So, it would appear that your configuration should successfully enable a message to be sent via Amazon SNS. This suggests that the problem lies with the HTTPS subscription, either from sending it from SNS or receiving it in the application.
I recommend that you add an Email or SMS subscription to verify whether Amazon SNS is receiving the topic and forwarding it to subscribers. If this works successfully, then you will need to debug the receipt of the message in the HTTPS application.
You must add TopicConfiguration
Read more about enable event notification
Related
I'm setting up a new app that will use a Cognito User Pool. I would like to send SMS messages to my users for authentication. There are plenty of examples out there of setting this up, however every one of them that I have seen attaches an inline policy to the IAM role used by Cognito to send messages to SNS that uses "*" as the policy resource.
I would prefer to limit this role to ONLY publishing to the SNS topic needed to send SMS messages, rather than giving it permissions to publish to any SNS topic in my account.
How do I find the ARN of the SNS topic that AWS uses for sending SMS messages in my account? I've been unable to find that information in the documentation ,and neither the web GUI nor the AWS CLI shows an SNS topic in my account that has any SMS-related subscriptions.
Cognito does not leverage a topic to publish SMS through SNS. Instead, it targets the destination phone number directly as the resource (as shown here). To keep Cognito from being able to publish to any of the topics in your account, you could try extending your IAM role policy with something like:
{
"Statement": [{
"Effect": "Deny",
"Action": "sns:Publish",
"Resource": "arn:aws:sns:*"
}]
}
which would still allow Cognito to publish directly to phone numbers (as the "resource" in that case would just be the phone number).
I have an SNS(encrypted with KMS) which is subscribed by two lambdas. I am trying to add a Redrive policy to one of the subscriptions. The DLQ in question is encrypted.
Adding Redrive policy is giving me an error 'Couldn't check Amazon SQS queue permissions. Make sure that the queue exists and that your account has permission to read the attributes of the queue. To allow an Amazon SNS topic to send messages to an Amazon SQS queue, you must create an Amazon SQS queue policy'
I have tried giving SNS decrypt permission on the DLQ Queue. But no luck. Any leads?
Seems like the functionality is working even if it shows an alert in console. The messages get sent to DLQ on lambda failures even though the console shows an alert.
I've reached out to AWS Support, and it's a known UI issue. And as of now, there is still no ETA confirmed for the fix.
To workaround, however, you could change SQS Access Policy Principal section from
"Principal": {
"Service": "sns.amazonaws.com"
},
to
"Principal": "*",
or to:
"Principal": {
"AWS": "*"
},
Presuming you have used the SQS queue policy from the AWS documentation with the principal "Service":"sns.amazonaws.com" then the error message is caused by the fact that the console uses an IAM policy simulation but no principal is being passed.
Changing the principal to "AWS":"*" will allow the policy simulation to succeed and therefore remove the error.
Using the wildcard is safe provided you have a condition which limits access from the specific SNS topic only, as outlined in the prerequisites in AWS's documentation.
I am trying to read message from an encrypted SQS. Objects are landed on an S3 Bucket -> Trigger S3 Event -> Message sent to SQS -> SQS triggers Lambda to Process.
I have got this working using an AWS managed CMK. However, I can't get this working using AWS owned CMK e.g. alias/aws/sqs.
The message just goes into messages in flight and does not invoke the Lambda functions.
As per the the AWS documentation here https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-server-side-encryption.html#sqs-encryption-what-does-sse-encrypt If you don't specify a custom CMK, Amazon SQS uses the AWS managed CMK for Amazon SQS. But we can't attach any policies against AWS owned CMK e.g.
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {
"Service": "<<service>>.amazonaws.com"
},
"Action": [
"kms:GenerateDataKey*",
"kms:Decrypt"
],
"Resource": "*"
}]
}
My question is: is it possible to use the AWS owned CMK on an SQS and have Lambda functions be able to read from that queue?
There is a section in the above URL called Enable Compatibility between AWS Services Such as Amazon CloudWatch Events, Amazon S3, and Amazon SNS and Encrypted Queues.
It mentions attaching a policy to the CMK. However, there is an option to use the alias/aws/sqs. I was wondering if I was missing something here.
I spoke with AWS and KMS AWS Managed Key would not work in this scenario. We can't change the key policy for KMS AWS Managed Keys, so wouldn't be possible for scenario: S3 Bucket -> Trigger S3 Event -> Message sent to SQS -> SQS triggers Lambda to Process
I used KMS AWS Customer Managed Key and it worked fine.
I'm using AWS Backup to backup my resources. I would like to get notifications from failed backups, but the only way to check the status of backups is from the AWS Backup service page - there is nothing AWS Backup related on Cloudwatch metrics, I was thinking of creating SNS-topic from Cloudwatch metric but that doesn't seem to be possible now?
Another question - would there be any way to get weekly report from AWS Backup, like "There are 25 resources currently being backed up, and from the last 7 days there is 175 restore points available"?
First of all you should create an SNS topic, add AWS Backup as a trusted entity in the resource-based policy of the SNS topic:
{
"Sid": "__console_pub_0",
"Effect": "Allow",
"Principal": {
"Service": "backup.amazonaws.com"
},
"Action": "SNS:Publish",
"Resource": "arn:aws:sns:us-west-2:{accountId}:test"
}
then turns on notifications for that topic and add BACKUP_JOB_COMPLETED event by the following AWS documentation:
Using Amazon SNS to Track AWS Backup Events.
Each time when AWS Backup job is completed or failed you will be informed to subscribed email address in SNS topic.
However, I can't find a way to customize notification.
I'm simply attempting to setup a budget alert buy am getting the error Invalid SNS topic ARN when attempting to specify an SNS ARN for the alert.
I followed the steps below:
Created a new SNS Topic with SMS as the protocol.
Created a subscription to the topic (a US phone number).
Tested the subscription by sending a text (this worked).
Copied the ARN under the ARN column on the Topics section in AWS SNS.
Checked Notify via Amazon Simple Notification Service (SNS) topic and entered the ARN from #4.
I get the error Invalid SNS topic ARN next to the SNS Topic ARN field, immediately after copying-and-pasting the ARN into the field.
What's up? I know it's the correct ARN because I copied it directly from the console. Also, as far as I understand, Budgets and their alerts aren't tied the a specific region, although SNS Topics are (the topic is in us-east-2 N. Virginia).
Please let me know if you need more specifics.
Make sure you have allowed budget.amazonaws.com to be able to publish messages to this SNS topic in the SNS topic's resource based policy.
Add the following statement to the policy
{
"Sid": "ExampleSid123456789012",
"Effect": "Allow",
"Principal": {
"Service": "budgets.amazonaws.com"
},
"Action": "SNS:Publish",
"Resource": "your topic ARN"
}
Hope it works!
I only got it fixed when I created a standard topic (FIFO didn't work).
I got the same problem before, but was able to solved by these steps,
"Please comply with SNS ARN format"
My workaround was
-
1. Create an SNS Topic in US-EAST-1 region
2. Just used Basic Access policy
3. (Optional) publish and subscribe field, I used the option : Only the specified AWS account, pasted my AWS account ID
4. Edit SNS Topic and Enable access policy,by adding the service budgets.amazonaws.com , in SNS:Publish action
just follow and copy the format in the picture below, Then saved
5. After successfully created the SNS topic , I reloaded the configure
alert page, then pasted the SNS topic ARN. No more Errors and can
already confirm budget,
You will receive a confirmation Email.
With this content
Congratulations, you have successfully verified your SNS topic for AWS Budgets! You can now receive your AWS Budgets notifications
through this SNS topic.
I had the same problem. Here is what I experienced.
Started to create AWS billing alert
Tried to use existing SNS topic, got the same error message you did. Interestingly if you have a syntax error in your SNS ARN you will get different error message 'Please comply with SNS ARN format'.
Thought that SNS topic should be in US-EAST region. Created one there just for billing alerts. Still same error message.
Realized that topic must have correct policy attached to it. Added policy statement similar to what #mmal suggested. Still same error message.
But then I just reloaded the page and tried to create billing alert from scratch.
And second time it worked!!
Just as a minor point to this, after following the above steps, it still was not working. I discovered that if you have "Encryption" enabled on the SNS topic it will not work. Disabling this on the topic and refreshing the Budget edit screen fixed it!
You need to add the policy just below "Statement": [ on the existing SNS Access policy.
{
"Sid": "AWSBudgets-notification-1",
"Effect": "Allow",
"Principal": {
"Service": "budgets.amazonaws.com"
},
"Action": "SNS:Publish",
"Resource": "SNS-policy-ARN"
},
Makesure not to miss the comma at the end.
SNS-policy-ARN is the ARN of the same SNS topic
If you're running this in a .net application, with your AWS credentials in your ~/.AWS/credentials file, ensure that the Region that is defined in your appsettings.json matches the region in which your SNS Topic exists.
{
"AWS": {
"Profile": "myProfile",
"Region": "eu-west-2"
},
...