I'm sending, to a AWS SNS topic, through my application, some messages.
Is there a way to debug if these messages are actually reaching SNS, without attaching any subscription?
My actual "way" to debug is subscribing a SQS queue to catch all messages and see if there's traffic on this SQS destination queue.
(setting this up was not also straight forward since after successful subscription of the SQS, I've found out that I had also to edit Access policies on SQS to allow SNS to reach out the queue other wise no feedback about SNS sending anything)
Adding the following on Access policies Statement JSON prop:
{
"Sid": "topic-subscription-arn:aws:sns:xx-xx-1:9999999999999:my-sns-notifier-topic",
"Effect": "Allow",
"Principal":
{
"AWS": "*"
},
"Action": "SQS:SendMessage",
"Resource": "arn:aws:sqs:xx-xx-1:9999999999999:my-sqs-queue-catcher",
"Condition":
{
"ArnLike":
{
"aws:SourceArn": "arn:aws:sns:xx-xx-1:9999999999999:my-sns-notifier-topic"
}
}
}
Any better idea?
If by "debug" then you mean read, then it is not possible to do this without attaching a subscription to your topic. If a message arrives for a topic that does not have any subscriptions, it will simply be thrown away by SNS as there is no where to deliver it.
Related
In an attempt to further tighten the security of our solution we are now looking at the used SNS topics and SQS queues. All our components live in the same AWS account.
For starters we want to restrict the access to the SQS queues based on IP. So only requests coming from our NAT Gateway IP will be allowed. We don't allow anonymous access to our SQS queues.
But there seems no way to achieve this as the creator of the SQS queues - the AWS account id - has access per default. So you can't create an effective permission for another user in the same AWS account id. As this newly created user, user2, will fall under the same AWS account id, with the same set of permissions.
Am I correct in my understanding that all users in the same AWS account id have access per default to all created SQS queues as long as their IAM policy permits it? And is my assumption right that the same behavior goes for the SNS topics?
Below is the policy I would like the implement. Beside this policy I have no other policies active for this SQS q. But it is not honoring the source IP condition. I still can connect from everywhere when I use a correct AWS access key/secret combination. Only when I set the AWS principal to * - everyone - the policy seems effective.
{
"Version": "2012-10-17",
"Id": "arn:aws:sqs:eu-west-1:4564645646464564:madcowtestqueue/SQSDefaultPolicy",
"Statement": [
{
"Sid": "Sid1589365989662",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::4564645646464564:user/user2"
},
"Action": [
"SQS:DeleteMessage",
"SQS:SendMessage",
"SQS:ReceiveMessage"
],
"Resource": "arn:aws:sqs:eu-west-1:143631359317:madcowtestqueue",
"Condition": {
"IpAddress": {
"aws:SourceIp": "1.1.1.1"
}
}
}
]
}
Reference:
Using identity-based policies with Amazon SQS - Amazon Simple Queue Service
Using identity-based policies with Amazon SNS - Amazon Simple Notification Service
Amazon SQS
Amazon SQS has the ability to define Amazon SQS policies. These policies can be used in addition to IAM policies to grant access to a queue.
For example, a policy can be added that permits anonymous access to a queue, which is useful for external applications to send messages to the queue.
Interestingly, these policies can also be used to control access to the queue by IP address.
To test this, I did the following:
Created an Amazon SQS queue
Used an Amazon EC2 instance to send a message to the queue -- Successful
Added the following policy to the SQS queue:
{
"Version": "2012-10-17",
"Id": "Queue1_Policy_UUID",
"Statement": [
{
"Sid": "Queue1_AnonymousAccess_AllActions_IPLimit_Deny",
"Effect": "Deny",
"Principal": "*",
"Action": "SQS:SendMessage",
"Resource": "arn:aws:sqs:ap-southeast-2:xxx:queue",
"Condition": {
"NotIpAddress": {
"aws:SourceIp": "54.1.2.3/32"
}
}
}
]
}
The IP address is that of my Amazon EC2 instance.
I then tried send a message to the queue again from the EC2 instance -- Successful
I then ran the identical command from my own computer -- Not successful
Therefore, it would appear that the SQS policy can override the permissions granted via IAM.
(Be careful... I added a policy that Denied sqs:* on the queue, and I wasn't able to edit the policy or delete the queue! I had to use the root account to delete it.)
Amazon SNS
I managed to achieve the same result with Amazon SNS using this access policy:
{
"Version": "2008-10-17",
"Id": "__default_policy_ID",
"Statement": [
{
"Sid": "__default_statement_ID",
"Effect": "Deny",
"Principal": {
"AWS": "*"
},
"Action": "SNS:Publish",
"Resource": "arn:aws:sns:ap-southeast-2:xxx:topic",
"Condition": {
"NotIpAddress": {
"aws:SourceIp": "54.1.2.3/32"
}
}
}
]
}
I set up Notification in CodeCommit on all events. The rule target SNS topic has a policy that allows principal *. However when I created a pull request. There is no event went through my SNS topic. I have a email subscription to the topic.
What have I missed and what is the possible causes?
I was having the same problem.
The solution is to give permission in SNS to receive notifications from CodeCommit. To do so you need to edit the SNS "Access policy" and add the following rule:
{
"Sid": "AWSCodeStarNotifications_publish",
"Effect": "Allow",
"Principal": {
"Service": [
"codestar-notifications.amazonaws.com"
]
},
"Action": "SNS:Publish",
"Resource": "arn:aws:sns:us-east-1:123456789012:codestar-notifications-MyTopicForNotificationRules"
}
Just make sure to edit the Resource property first.
Source, full explanation and example here: https://docs.aws.amazon.com/codestar-notifications/latest/userguide/notification-target-create.html
I am struggling with what I would think would be a simple task.
I want to configure my SQS queue to allow S3 buckets in my account to send messages, but disallow outsiders to send messages. (Outsiders means any principal that is not a member of my AWS account)
The only SQS permission configuration that I can get to work is Effect=Allow, Principals=*, Actions=SQS:SendMessage, Conditions=None
Any other permission causes me to see this error when I create the [S3 event -> SQS]: Unable to validate the following destination configurations. Permissions on the destination queue do not allow S3 to publish notifications from this bucket.
Principals=* concerns me. From the documentation I can find, this means that the SQS queue is accessible from anyone in the world. Is this true? This is obviously very bad.
How can I allow my S3 buckets to SendMessage to my SQS queue, and not anonymous users to push messages?
It is acceptable to me to allow any resource in my AWS account to SendMessage to SQS. I just need to block access to anonymous AWS users. This is a very basic requirement and I'm very surprised that I can't find a simple way to do this.
You can find the secure configuration right in the document
"Condition": {
"ArnLike": { "aws:SourceArn": "arn:aws:s3:*:*:bucket-name" }
}
Note that for both the Amazon SNS and Amazon SQS IAM policies, you can specify the StringLike condition in the policy, instead of the ArnLike condition.
"Condition": {
"StringLike": { "aws:SourceArn": "arn:aws:s3:*:*:bucket-name" }
}
Full example pulled from the doc
{
"Sid": "example-statement-ID",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"SQS:SendMessage"
],
"Resource": "SQS-ARN",
"Condition": {
"ArnLike": { "aws:SourceArn": "arn:aws:s3:*:*:bucket-name" }
}
}
I am a beginner to AWS CloudWatch. The event is not getting triggered when I use AWS java SDK to create CloudWatch event rules, and using sns topic as a target.
It's working fine when created using Direct AWS management console.
Everything remains the same when comparing java sdk creation and management console creation.
The only difference is in aws management console rules invoke, two metrics are created(invocation, TriggeredRules), in java sdk rules invoke, three metrics are created(invocation, TriggeredRules,FailedInvocation).
If you use a custom KMS key on your SNS Topic, you need also add the following policy to your KMS key policy:
{
"Sid": "CloudwatchEvents",
"Effect": "Allow",
"Principal": {
"Service": "events.amazonaws.com"
},
"Action": [
"kms:Encrypt*",
"kms:Decrypt*",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:Describe*"
],
"Resource": "*"
}
If you find that it works when created via the console but not if you do it with the API (or something like Terraform) then it is likely that you are not updating the SNS Topic Policy so that it allows events to be published from CloudWatch Events. The console does this for you semi-magically but if you use the APIs you have a bit more work to do.
There is an answer here in the FAQ with the details but the long and short of it is you need to add (not replace) something like this to your SNS Topic Policy:
{
"Sid" : "CloudWatchEvents",
"Effect" : "Allow",
"Resource" : "${aws_sns_topic.events.arn}",
"Action" : "sns:Publish",
"Principal" : {
"Service" : "events.amazonaws.com"
}
}
I have a SNS topic and subscriber set up like so :
The subscribers are correctly notified when the Topic is tested via "Publish to Topic" from AWS console
I have a IoT rule like so :
I have a policy attached to the rule like so :
{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Action": "sns:Publish",
"Resource": "arn:aws:sns:ap-southeast-xxxxxx:MySNSTopic"
}
}
Yet when I try and test from the MQTT browser client, the notification is not fired
What am I missing?
Started working after a few minutes. Guess it needs some time to set up