I have 2 different buckets.
One is able to attach Amazon SQS to event notification and the second doesn't.
SQS permissions is broad enough. It's smth with S3 bucket. But I can't figure it out.
There are no "Deny" clauses in bucket policy.
There are several additional ACL though...
This is my Access Policy stored in SQS:
{
"Version": "2008-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": "*",
"Resource": "*"
}
]
}
This is the error:
Yes, I have seen this before.
You must create an Access Policy on the Amazon SQS queue to permit access by the Amazon S3 bucket.
Here is a sample policy from Granting permissions to publish event notification messages to a destination - Amazon Simple Storage Service:
{
"Version": "2012-10-17",
"Id": "example-ID",
"Statement": [
{
"Sid": "example-statement-ID",
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": [
"SQS:SendMessage"
],
"Resource": "arn:aws:sqs:Region:account-id:queue-name",
"Condition": {
"ArnLike": { "aws:SourceArn": "arn:aws:s3:*:*:awsexamplebucket1" },
"StringEquals": { "aws:SourceAccount": "bucket-owner-account-id" }
}
}
]
}
See also: Walkthrough: Configuring a bucket for notifications (SNS topic or SQS queue) - Amazon Simple Storage Service
I contacted AWS Support. There provided undocumented details:
So, each time an S3 event is edited or saved, there will be validation
of the current destinations since the PutBucketNotification will
replace the existing notification configuration and not update it.
So I had another legacy S3 event that didn't have correct permissions. I removed it, and everything is working right now. Thank you to all who tried to help :)
Related
I have a SQS setup in AWS with an access policy which allows another account's SNS to push message,
and an instance with an IAM role allowing the instance to communicate with the queue.
I found that if I apply the access policy of SQS, the instance cannot access the SQS. Removing it, the instance can work with the queue.
Is there an explanation which can explan this?
Remove the access policy from the queue, works;
SQS Policy as following:
{
"Version": "2012-10-17",
"Id": "sqspolicy",
"Statement": [
{
"Sid": "First",
"Effect": "Allow",
"Principal": {
"Service": "sns.amazonaws.com"
},
"Action": "sqs:SendMessage",
"Resource": "${module.elmo-s-rem-sqs-user-sync-service.primary_queue_arn}",
"Condition": {
"ArnEquals": {
"aws:SourceArn": "${local.pivot_topic_arn}"
}
}
}
]
}
Have a tried by removing the Principal part, that could let the simulator pass. But the sns couldn't publish message any more.
I am trying deliver a message from an unencrypted SNS topic to an encrypted SQS queue following this guide. I was able to complete the "Configure KMS permissions for AWS services" step, but I am having trouble with the "Configure KMS permissions for producers
" step. I have created the IAM role, however attaching this role to my SNS topic is where I am specifically confused. Here are some questions I have which my own research was unable to answer:
Can an IAM role be attached to a specific item (SNS topic, SQS queue, etc...)? If not, what other way is there to grant permissions to a specific item?
When the instructions mention "producer", is this referring to the SNS topic or the AWS account which owns the SNS topic?
Any and all help is greatly appreciated.
Edit:
Here is my current AWS KMS key policy:
{
"Version": "2012-10-17",
"Id": "key-consolepolicy-3",
"Statement": [
{
"Sid": "Allow administration of the key",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${aws_account_id}:root"
},
"Action": "kms:*",
"Resource": "*"
},
{
"Sid": "Allow SNS to use KMS",
"Effect": "Allow",
"Principal": {
"Service": "sns.amazonaws.com"
},
"Action": [
"kms:GenerateDataKey",
"kms:Decrypt"
],
"Resource": "*"
}
]
}
Whenever I add the following statements to my KMS key policy in the Statement list, I get the error "MalformedPolicyDocumentException - Policy contains a statement with no principal":
{
"Effect": "Allow",
"Action": [
"kms:GenerateDataKey",
"kms:Decrypt"
],
"Resource": "${kms_customer_managed_key_arn}"
},
{
"Effect": "Allow",
"Action": [
"sqs:SendMessage"
],
"Resource": "${sqs_queue_arn}"
}
If not, what other way is there to grant permissions to a specific item?
The permissons from the link are attached to your KMS CMK key policy.
When the instructions mention "producer", is this referring to the SNS topic or the AWS account which owns the SNS topic?
The producer is anyone or anything that sends messages. It can be a lambda function, an ec2 instance or IAM user/role. In that case you give the producer permissions to sendMessage and use the KMS key. For lambda it would be in lambda execution role, for instance it would be in an instance role.
I have a NodeJS application that publishes message to AWS SNS topic string and a AWS SQS subscription for the same. On the SQS console, I can see the published message. However, I am not clear with the access policy of the SQS queue.
This answer mentions the use of "Principal": "*" - but, that is very broad. One could probably use "Principal" : {"AWS": "*"}; but, that isn't narrow either.
{
"Version": "2012-10-17",
"Id": "Policy1607949016538",
"Statement": [
{
"Sid": "Stmt1607949012567",
"Effect": "Allow",
"Principal": "*",
"Action": [
"sqs:ReceiveMessage",
"sqs:SendMessage"
],
"Resource": "arn:aws:sqs:ap-south-1:463113000000:orders"
}
]
}
Questions
While delivering a message to SQS queue, as a result of subscription, which user is in effect? Same as the one who published to the topic?
I could get the messages to flow into the queue only when I used "Principal" : {"AWS": "*"}. So, how should I define a restrictive policy such that messages are written to queues only as a result of subscription?
What is the equivalent in the AWS SQS CLI to create a queue with "Principal" : {"AWS": "*"} permissions?
The only user that matters is the one that qualifies for the policy as defined for subscription and SQS access policy.
The Condition in policy document can make the overall policy restrictive. See example below.
Adding SQS Permissions with conditions using AWS CLI Command
Example policy document restricting access to account ID.
{
"Version": "2012-10-17",
"Id": "Policy1607960702002",
"Statement": [
{
"Sid": "Stmt1607960701004",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"sqs:ReceiveMessage",
"sqs:SendMessage"
],
"Resource": "arn:aws:sqs:ap-south-1:463113000000:orders",
"Condition": {
"StringEquals": {
"aws:SourceAccount": "463113000000"
}
}
}
]
}
I have an SQS event set for S3 bucket.When a file with name xyz.txt is uploaded in S3, SQS event is called and then this SQS calls an lambda.
This workflow is working fine.
Now I want to encrypt my SQS Queue for this I am using one of existing CMK "services-cmk". After queue is encrypted S3 is not able to invoke SQS. When I am putting message directly in encrypted SQS its working and lambda also got triggered.
I checked policy of services-cmk and it allows S3.I am not able to find out what I am missing here ?
You have to explicitly specify S3 (using root as Principal is not enough) in your KMS key policy as explained here:
{
"Version": "2012-10-17",
"Id": "example-ID",
"Statement": [
{
"Sid": "example-statement-ID",
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": [
"kms:GenerateDataKey",
"kms:Decrypt"
],
"Resource": "*"
}
]
}
I keep getting this error, when I try to configure the S3 bucket to write the response to SQS.
Is there any way I can solve this?
S3/SQS integration does not currently support the use of FIFO queues, and your queue is a FIFO queue.
The following features of AWS services aren't currently compatible with FIFO queues:
Amazon CloudWatch Events
Amazon S3 Event Notifications
Amazon SNS Topic Subscriptions
https://aws.amazon.com/sqs/faqs/#fifo-queues
Did you attach a policy like this to your SQS queue?
{
"Version": "2008-10-17",
"Id": "example-ID",
"Statement": [
{
"Sid": "AllowS3ToPublishMessages",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"SQS:SendMessage"
],
"Resource": "sqs-episource-arn",
"Condition": {
"ArnLike": {
"aws:SourceArn": "arn:aws:s3:*:*:your-bucket-name"
}
}
}
]
}
(replacing sqs-episource-arn with the full ARN for EpisourceExp.fifo and your-bucket-name with the bucket name of your S3 bucket)