I have an SQS event set for S3 bucket.When a file with name xyz.txt is uploaded in S3, SQS event is called and then this SQS calls an lambda.
This workflow is working fine.
Now I want to encrypt my SQS Queue for this I am using one of existing CMK "services-cmk". After queue is encrypted S3 is not able to invoke SQS. When I am putting message directly in encrypted SQS its working and lambda also got triggered.
I checked policy of services-cmk and it allows S3.I am not able to find out what I am missing here ?
You have to explicitly specify S3 (using root as Principal is not enough) in your KMS key policy as explained here:
{
"Version": "2012-10-17",
"Id": "example-ID",
"Statement": [
{
"Sid": "example-statement-ID",
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": [
"kms:GenerateDataKey",
"kms:Decrypt"
],
"Resource": "*"
}
]
}
Related
I'm using the AWS SQS service, and I'm having a hard time defining permissions on my SQS queue. In my setup I'm using the AWS Lambda service, which is triggered when an object is pushed onto an S3 bucket.
However to keep my question briefly, this is what I want to achieve:
Object is pushed to a S3 bucket,
S3 bucket triggers AWS Lambda,
Lambda does some calculations and pushes an event to my SQS queue (Permission needs to be defined)
Application reads from SQS
Lambda and SQS queue are in different AWS account-
Steps followed-
Added permission for access role assumed by lambda in SQS access policy-
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::accountid:role/rolename",
]
},
"Action": "SQS:SendMessage",
"Resource": "https://sqs.us-east-1.amazonaws.com/accountid/qsqqueuename"
}
]
}
SQS queue has KMS key enabled so gave permission in kms access policy to the same role
The role assumed by lambda has following access-
{
"Action": [
"s3:PutObject",
"s3:GetObject",
"kms:Decrypt",
"kms:Encrypt",
"sqs:SendMessage",
"kms:DescribeKey",
"s3:ListBucket",
"ssm:GetParameter",
"kms:ReEncrypt*",
"kms:GenerateDataKey*"
],
"Resource": [
"arn:aws:kms:us-east-1:accountid:key/kmskey5809e1338be5",
"arn:aws:sqs:us-east-1:accountid:sqaqueuename"
],
"Effect": "Allow",
"Sid": "mailboxaccess"
},
My lambda is giving the error-
An error occurred (AccessDenied) when calling the SendMessage operation: Access to the resource https://queue.amazonaws.com/ is denied.
Any suggestions?
The problem was the arn for sqs..I had provided an HTTP address for the queue whereas need to provide an arn.
I have a SQS setup in AWS with an access policy which allows another account's SNS to push message,
and an instance with an IAM role allowing the instance to communicate with the queue.
I found that if I apply the access policy of SQS, the instance cannot access the SQS. Removing it, the instance can work with the queue.
Is there an explanation which can explan this?
Remove the access policy from the queue, works;
SQS Policy as following:
{
"Version": "2012-10-17",
"Id": "sqspolicy",
"Statement": [
{
"Sid": "First",
"Effect": "Allow",
"Principal": {
"Service": "sns.amazonaws.com"
},
"Action": "sqs:SendMessage",
"Resource": "${module.elmo-s-rem-sqs-user-sync-service.primary_queue_arn}",
"Condition": {
"ArnEquals": {
"aws:SourceArn": "${local.pivot_topic_arn}"
}
}
}
]
}
Have a tried by removing the Principal part, that could let the simulator pass. But the sns couldn't publish message any more.
I have 2 different buckets.
One is able to attach Amazon SQS to event notification and the second doesn't.
SQS permissions is broad enough. It's smth with S3 bucket. But I can't figure it out.
There are no "Deny" clauses in bucket policy.
There are several additional ACL though...
This is my Access Policy stored in SQS:
{
"Version": "2008-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": "*",
"Resource": "*"
}
]
}
This is the error:
Yes, I have seen this before.
You must create an Access Policy on the Amazon SQS queue to permit access by the Amazon S3 bucket.
Here is a sample policy from Granting permissions to publish event notification messages to a destination - Amazon Simple Storage Service:
{
"Version": "2012-10-17",
"Id": "example-ID",
"Statement": [
{
"Sid": "example-statement-ID",
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": [
"SQS:SendMessage"
],
"Resource": "arn:aws:sqs:Region:account-id:queue-name",
"Condition": {
"ArnLike": { "aws:SourceArn": "arn:aws:s3:*:*:awsexamplebucket1" },
"StringEquals": { "aws:SourceAccount": "bucket-owner-account-id" }
}
}
]
}
See also: Walkthrough: Configuring a bucket for notifications (SNS topic or SQS queue) - Amazon Simple Storage Service
I contacted AWS Support. There provided undocumented details:
So, each time an S3 event is edited or saved, there will be validation
of the current destinations since the PutBucketNotification will
replace the existing notification configuration and not update it.
So I had another legacy S3 event that didn't have correct permissions. I removed it, and everything is working right now. Thank you to all who tried to help :)
I am trying deliver a message from an unencrypted SNS topic to an encrypted SQS queue following this guide. I was able to complete the "Configure KMS permissions for AWS services" step, but I am having trouble with the "Configure KMS permissions for producers
" step. I have created the IAM role, however attaching this role to my SNS topic is where I am specifically confused. Here are some questions I have which my own research was unable to answer:
Can an IAM role be attached to a specific item (SNS topic, SQS queue, etc...)? If not, what other way is there to grant permissions to a specific item?
When the instructions mention "producer", is this referring to the SNS topic or the AWS account which owns the SNS topic?
Any and all help is greatly appreciated.
Edit:
Here is my current AWS KMS key policy:
{
"Version": "2012-10-17",
"Id": "key-consolepolicy-3",
"Statement": [
{
"Sid": "Allow administration of the key",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${aws_account_id}:root"
},
"Action": "kms:*",
"Resource": "*"
},
{
"Sid": "Allow SNS to use KMS",
"Effect": "Allow",
"Principal": {
"Service": "sns.amazonaws.com"
},
"Action": [
"kms:GenerateDataKey",
"kms:Decrypt"
],
"Resource": "*"
}
]
}
Whenever I add the following statements to my KMS key policy in the Statement list, I get the error "MalformedPolicyDocumentException - Policy contains a statement with no principal":
{
"Effect": "Allow",
"Action": [
"kms:GenerateDataKey",
"kms:Decrypt"
],
"Resource": "${kms_customer_managed_key_arn}"
},
{
"Effect": "Allow",
"Action": [
"sqs:SendMessage"
],
"Resource": "${sqs_queue_arn}"
}
If not, what other way is there to grant permissions to a specific item?
The permissons from the link are attached to your KMS CMK key policy.
When the instructions mention "producer", is this referring to the SNS topic or the AWS account which owns the SNS topic?
The producer is anyone or anything that sends messages. It can be a lambda function, an ec2 instance or IAM user/role. In that case you give the producer permissions to sendMessage and use the KMS key. For lambda it would be in lambda execution role, for instance it would be in an instance role.
I keep getting this error, when I try to configure the S3 bucket to write the response to SQS.
Is there any way I can solve this?
S3/SQS integration does not currently support the use of FIFO queues, and your queue is a FIFO queue.
The following features of AWS services aren't currently compatible with FIFO queues:
Amazon CloudWatch Events
Amazon S3 Event Notifications
Amazon SNS Topic Subscriptions
https://aws.amazon.com/sqs/faqs/#fifo-queues
Did you attach a policy like this to your SQS queue?
{
"Version": "2008-10-17",
"Id": "example-ID",
"Statement": [
{
"Sid": "AllowS3ToPublishMessages",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"SQS:SendMessage"
],
"Resource": "sqs-episource-arn",
"Condition": {
"ArnLike": {
"aws:SourceArn": "arn:aws:s3:*:*:your-bucket-name"
}
}
}
]
}
(replacing sqs-episource-arn with the full ARN for EpisourceExp.fifo and your-bucket-name with the bucket name of your S3 bucket)