Unable to configure SQS queue notification in S3 - amazon-web-services

I created an SQS queue and added policy under permission tab allowing only my account users to configure the configure the notification
Policy Document
{
"Version": "2012-10-17",
"Id": "arn:aws:sqs:us-east-1:111111111111:sqsqueue/SQSDefaultPolicy",
"Statement": [
{
"Sid": "Sid111111111111",
"Effect": "Allow",
"Principal": {
"AWS": "111111111111"
},
"Action": [
"sqs:SendMessage",
"sqs:ReceiveMessage"
],
"Resource": "arn:aws:sqs:us-east-1:111111111111:queue"
}
]
Navigate to S3 and try to configure event notification for the above queue, it is throwing an error
Unable to validate the following destination configurations. Permissions on the destination queue do not allow S3 to publish
notifications from this bucket.
(arn:aws:sqs:us-east-1:111111111111:queue)*
am I doing something wrong? Can someone help me please

I was able to resolve this issue by adding "Service": "s3.amazonaws.com"
in the Principal tag.
Here the policy document
{
"Version": "2012-10-17",
"Id": "arn:aws:sqs:us-east-1:111111111111:sqsqueue/SQSDefaultPolicy",
"Statement": [
{
"Sid": "Sid111111111111",
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": [
"sqs:SendMessage",
"sqs:ReceiveMessage"
],
"Resource": "arn:aws:sqs:us-east-1:111111111111:queue"
}
]
This is explained in https://forums.aws.amazon.com/thread.jspa?threadID=173251

This template file creates a bucket, SQS Queue and a policy to connect the two:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
IncomingBucketName:
Type: 'String'
Description: 'Incoming Bucket Name'
Default: 'some-bucket-name-here'
Resources:
IncomingFileQueue:
Type: 'AWS::SQS::Queue'
Properties: {}
SQSQueuePolicy:
Type: 'AWS::SQS::QueuePolicy'
Properties:
PolicyDocument:
Id: 'MyQueuePolicy'
Version: '2012-10-17'
Statement:
- Sid: 'Statement-id'
Effect: 'Allow'
Principal:
AWS: "*"
Action: 'sqs:SendMessage'
Resource:
Fn::GetAtt: [ IncomingFileQueue, Arn ]
Queues:
- Ref: IncomingFileQueue
IncomingFileBucket:
Type: 'AWS::S3::Bucket'
DependsOn:
- SQSQueuePolicy
- IncomingFileQueue
Properties:
AccessControl: BucketOwnerFullControl
BucketName:
Ref: IncomingBucketName
NotificationConfiguration:
QueueConfigurations:
- Event:
s3:ObjectCreated:Put
Queue:
Fn::GetAtt: [ IncomingFileQueue, Arn ]
I was getting the same issue but used this page to work out how to connect the three resources in order to successfully deploy the stack:
https://aws.amazon.com/premiumsupport/knowledge-center/unable-validate-destination-s3/
I'm still working on the Policy Condition as the form recommended in the above link doesn't work for SQS. That being the case, the above template is not secure and shouldn't be used in production as it allows anyone to add messages to the queue.
I'll update this answer once I've figured that bit out...

Related

CloudFormation API Gateway Lambda integration request not getting right permissions?

I have a CloudFormation stack template that creates an API Gateway resource where the method is of Type: LAMBDA_PROXY. It initially works fine for accessing the root domain, for example https://28af295f439b5f0aef7c7805864ba3981f282e1e.guacchain.com/ - but when I try accessing https://28af295f439b5f0aef7c7805864ba3981f282e1e.guacchain.com/about the network request gives back status code 500 and response {"message": "Internal server error"}
The generated Lambda function has this as its Resource-based Policy:
{
"Version": "2012-10-17",
"Id": "default",
"Statement": [
{
"Sid": "stack-28af295f439b5f0aef7c7805864ba3981f282e1e-lambdaApiGatewayInvoke-128TRSSUE8WDQ",
"Effect": "Allow",
"Principal": {
"Service": "apigateway.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:us-east-1:378688096774:function:lambda-28af295f439b5f0aef7c7805864ba3981f282e1e",
"Condition": {
"ArnLike": {
"AWS:SourceArn": "arn:aws:execute-api:us-east-1:378688096774:bccwb0kvvd/*/*/*"
}
}
}
]
}
When going to the API Gateway Integration Request page and simply editing the Lambda Function field (cutting the existing value then pasting it back, then hitting the checkmark), I get this "Add permission" popup:
After clicking OK and then refreshing the Lambda console page, its Resource-base Policy updates to include two seemingly duplicate statements (only difference being the Sid field):
{
"Version": "2012-10-17",
"Id": "default",
"Statement": [
{
"Sid": "stack-28af295f439b5f0aef7c7805864ba3981f282e1e-lambdaApiGatewayInvoke-128TRSSUE8WDQ",
"Effect": "Allow",
"Principal": {
"Service": "apigateway.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:us-east-1:378688096774:function:lambda-28af295f439b5f0aef7c7805864ba3981f282e1e",
"Condition": {
"ArnLike": {
"AWS:SourceArn": "arn:aws:execute-api:us-east-1:378688096774:bccwb0kvvd/*/*/*"
}
}
},
{
"Sid": "d6d795d4-8461-4774-bd6e-ae8d8ea3bcee",
"Effect": "Allow",
"Principal": {
"Service": "apigateway.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:us-east-1:378688096774:function:lambda-28af295f439b5f0aef7c7805864ba3981f282e1e",
"Condition": {
"ArnLike": {
"AWS:SourceArn": "arn:aws:execute-api:us-east-1:378688096774:bccwb0kvvd/*/*/*"
}
}
}
]
}
After deploying the API Gateway and waiting about a minute, https://28af295f439b5f0aef7c7805864ba3981f282e1e.guacchain.com/about finally becomes accessible. So my question is, what's inadequate with the original Lambda Resource-based Policy that prevents all requests except for the / one to be accessed on the domain?
One subtle note I want to point out is that after cutting the Lambda function name and pasting it on the Integration Request page, it doesn't show as an autocomplete option, while others do.
This is the lambdaIAMRole I have defined in the CloudFormation stack:
lambdaIAMRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Action:
- 'sts:AssumeRole'
Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Policies:
- PolicyDocument:
Version: 2012-10-17
Statement:
- Action:
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
Effect: Allow
Resource:
- !Sub >-
arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/${lambdaFunctionName}:*
PolicyName: lambda
And the Lambda function resource has Role: !GetAtt lambdaIAMRole.Arn as part of its Properties field.
Based on the comments and inspection of the CloudFormation template, it was found that the issue was caused by the incorrect setup of IntegrationHttpMethod in apiGatewayLambdaResourceMethod.
Instead of
IntegrationHttpMethod: GET
it should be
IntegrationHttpMethod: POST
This is because AWS_PROXY for lambda requires POST method, not GET.

How to create a private AWS Api Gateway using cloudformation?

I am trying to create an AWS API Gateway of PRIVATE type,
This requires a resource policy, which I have as I'm able to create the gateway from the AWS Console,
I wanted to know how I could add the resource policy via the CF template -
Following is the swagger definition of the resource policy -
x-amazon-apigateway-policy:
Version: "2012-10-17"
Statement:
- Effect: "Deny"
Principal: "*"
Action: "execute-api:Invoke"
Resource: "arn:aws:execute-api:us-east-1:awsAccountId:xxxx/*/*/*"
Condition:
StringNotEquals:
aws:sourceVpc: "vpc-xxxxx"
- Effect: "Allow"
Principal: "*"
Action: "execute-api:Invoke"
Resource: "arn:aws:execute-api:us-east-1:xxxx:xxxx/*/*/*"
How can I configure it in the CF template -
AWSTemplateFormatVersion: 2010-09-09
Transform: 'AWS::Serverless-2016-10-31'
Description: G2G Api Template Stack
Resources:
g2gPrivate:
Type: 'AWS::ApiGateway::RestApi'
Properties:
Name: 'private-gw'
EndpointConfiguration:
Types:
- PRIVATE
Reference -
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-apigateway-restapi.html
https://medium.com/#cathmgarcia/conditional-resource-policy-on-aws-sam-with-inline-swagger-816ce946dbb
You need to supply the policy under a key (called Policy at the same level as Name.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-apigateway-restapi.html#cfn-apigateway-restapi-policy
This needs to be supplied in the JSON format.
Something like...
AWSTemplateFormatVersion: 2010-09-09
Transform: 'AWS::Serverless-2016-10-31'
Description: G2G Api Template Stack
Resources:
g2gPrivate:
Type: 'AWS::ApiGateway::RestApi'
Properties:
Name: 'private-gw'
EndpointConfiguration:
Types:
- PRIVATE
Policy: !Sub |
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Deny",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "arn:aws:execute-api:us-east-1:${AWS::AccountId}:*/*/*/*",
"Condition": {
"StringNotEquals": {
"aws:sourceVpc": "vpc-xxxxx"
}
}
},
{
"Effect": "Allow",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "arn:aws:execute-api:us-east-1:${AWS::AccountId}:*/*/*/*"
}
]
}

Message isn't sent to SQS when I specify Principal in access policy

I have an infrastructure where SNS topic sends messages to SQS (using SNS subscription of course). When I setup the following access policy it works.
{
"Version": "2012-10-17",
"Id": "__default_policy_ID",
"Statement": [
{
"Sid": "SendMessagePolicy",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "SQS:SendMessage",
"Resource": "arn:aws:sqs:us-east-1:312226948869:mr-sandbox-loyalty",
"Condition": {
"ArnEquals": {
"AWS:SourceArn": "arn:aws:sns:us-east-1:312226948869:mr-sandbox-transaction-created"
}
}
}
]
}
BUT when instead of * I setting up arn:aws:iam::312226948869:root messages aren't sent to queue. The account number which I used is 312226948869. Any ideas?
Thanks.
UPDATE
In web console, when I'm trying to set Principal: 312226949769 it's overrided as Principal: arn:aws:iam::312226949769:root
I managed to fix the issue. I added IAM ARN to ArnEquals condition.
{
"Version": "2012-10-17",
"Id": "__default_policy_ID",
"Statement": [
{
"Sid": "SendersPolicy",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "sqs:SendMessage",
"Resource": "arn:aws:sqs:us-east-1:%account_id%:mr-prod-loyalty-program",
"Condition": {
"ArnEquals": {
"AWS:SourceArn": [
"arn:aws:iam::%account_id%:role/boss-mr-prod-sqs-dashboard",
"arn:aws:iam::%account_id%:role/IDT-PSF-Instance-Profile"
]
}
}
}
]
}
My solution ended up not using my specific account as principal but instead service of SNS. This should be fine since I have the condition of specific sns topic arn
Resources:
Policy:
Type: AWS::SQS::QueuePolicy
Properties:
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- sns.amazonaws.com
Action:
- sqs:SendMessage
Resource: !GetAtt MySQS.Arn
Condition:
ArnEquals:
aws:SourceArn: !Ref MyTopic
Queues:
- !Ref MySQS

Creating CloudWatch rule with AWS CloudFormation not linking to role

I am trying to create a CloudWatch rule that triggers on a schedule and executes a state machine (Step Functions). I'm using CloudFormation to create this, and everything creates fine except for the association of the IAM role used by the rule, to the rule itself. Here is what I mean:
Notice under 'Use Existing Role' it's blank.
Here is the CF template portion that deals with the rule and its role.
"SFInvoke":{
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": {
"Fn::Sub": "states.${AWS::Region}.amazonaws.com"
}
},
"Action": "sts:AssumeRole"
}
]
},
"Policies": [
{
"PolicyName": "StepFunctionsInvoke",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"states:StartExecution"
],
"Resource": { "Ref" : "StateMachine"}
}
]
}
}
]
}
},
"CloudWatchStateMachineSDCEventRule": {
"Type":"AWS::Events::Rule",
"Properties": {
"Description":"CloudWatch trigger for the InSite Static Data Consumer",
"ScheduleExpression": "rate(5 minutes)",
"State":"ENABLED",
"Targets":[{
"Arn":{ "Ref" : "StateMachine"},
"Id":"StateMachineTargetId",
"RoleArn":{
"Fn::GetAtt": [
"SFInvoke",
"Arn"
]
}
}]
}
},
You want the SFInvoke role to show up on the Use existing role selector?
If that is the case, you need to set the Principal to events instead of states.
You're editing the event target in the screenshot above, not the step function. Principal defines the service that can assume the role, in your case that is the events service.
Try this for role creation:
"SFInvoke":{
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "events.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
},
"Policies": [
{
"PolicyName": "StepFunctionsInvoke",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"states:StartExecution"
],
"Resource": { "Ref" : "StateMachine"}
}
]
}
}
]
}
}
Probably the Yaml would be as:
Based on the Principal: as an Events Based Services and Actions: to start the execution of a StepFunctions State Machine.
AWSEventsInvokeStepFunctions:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: AWSEventsInvokeStepFunctions
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- states:StartExecution
Resource: !Sub "arn:aws:states:${AWS::Region}:${AWS::AccountId}:stateMachine:*"
The Role which is now generic in nature can be applied to a CloudWatch Event Rule, gives Rule with the permissions to be able to start the Execution of a StepFunctions State Machine based on an Amazon S3 Event.
AmazonCloudWatchEventRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.s3
detail-type:
- 'AWS API Call via CloudTrail'
detail:
eventSource:
- s3.amazonaws.com
eventName:
- PutObject
requestParameters:
bucketName:
- !Ref EventBucket
Targets:
-
RoleArn: !GetAtt AWSEventsInvokeStepFunctions.Arn
Arn: !Sub "arn:aws:states:${AWS::Region}:${AWS::AccountId}:stateMachine:MyStateMachine"
Id: !Sub "StepExecution"
You can probably check more on Start the Execution of State Machine based on Amazon S3 Event

Setting up cross account access of IAM user/role

I have a main account user that I want to allow access to a subaccount S3 bucket. I have setup the following stack in my subaccount
AWSTemplateFormatVersion : '2010-09-09'
Description: 'Skynet stack to allow admin account deploying user to access S3'
Parameters:
AccountId:
Type: String
Description: Account ID of admin account (containing user to allow)
Username:
Type: String
Description: Username to be allowed access
BucketPrefix:
Type: String
Description: Bucket to be allowed (prefix appended with -{AccountId}-{Region})
Resources:
CrossAccountRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Action: sts:AssumeRole
Principal:
AWS:
- !Sub arn:aws:iam::${AccountId}:user/${Username}
Path: /
Policies:
- PolicyName: skynet-s3-delegate
PolicyDocument:
Statement:
- Effect: Allow
Action:
- s3:ListBucket
- s3:GetObject
Resource: "*"
But I find that I still get an error when I try to assume the role:
aws s3 cp skynet-lambda.zip s3://skynet-lambda-TARGET_ACCOUNT_ID-ap-southeast-1 --profile skynetci-cross-account
An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:iam::MAIN_ACCOUNT_ID:user/circleci-skynet is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::TARGET_ACCOUNT_ID:role/StackSet-df0e85b0-d6fd-47bf-a0bb-CrossAccountRole-1EW45TXEFAY0D
Why is this so considering I already have the following policy for the user
{
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": "arn:aws:iam::TARGET_ACCOUNT_ID:role/StackSet-df0e85b0-d6fd-47bf-a0bb-CrossAccountRole-1EW45TXEFAY0D"
}
You need to have Bucket Policy update to allow cross account access a sample policy would be like:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Example permissions",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountB-ID:root"
},
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::examplebucket"
]
}
]
}
Also make sure IAM user who is trying to access has this inline policy attached:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Example",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::examplebucket"
]
}
]
}
You can refer AWS Documentation
I am attaching a working example that I tested using two of my accounts.
STEP 1: CloudFormation YAML template:
AWSTemplateFormatVersion : '2010-09-09'
Description: 'Skynet stack to allow admin account deploying user to access S3'
Parameters:
AccountId:
Type: String
Description: Account ID of admin account (containing user to allow)
Username:
Type: String
Description: Username to be allowed access
BucketPrefix:
Type: String
Description: Bucket to be allowed (prefix appended with -{AccountId}-{Region})
Resources:
CrossAccountRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Action: sts:AssumeRole
Principal:
AWS:
- !Sub arn:aws:iam::${AccountId}:user/${Username}
Path: /
Policies:
- PolicyName: skynet-s3-delegate
PolicyDocument:
Statement:
- Effect: Allow
Action:
- s3:ListBucket
- s3:GetObject
Resource: "*"
RootInstanceProfile:
Type: "AWS::IAM::InstanceProfile"
Properties:
Path: "/"
Roles:
-
Ref: "CrossAccountRole"
STEP 2: Create the cross account profile
Modify ~/.aws/credentials. Add a new profile called "skynetci-cross-account". Modify based upon your parameters created in STEP 1. You will need the role arn to replace the one below. You will also need the profile name for the account that you are giving permission to. In this example the profile name is "default".
Example here:
[skynetci-cross-account]
role_arn = arn:aws:iam::191070ABCDEF:role/Test-CrossAccountRole-IZDDLRUMABCD
source_profile = default
STEP 3: Test cross access
aws --profile skynetci-cross-account s3 ls s3://bucket-name
To accomplish your goal, you need to set a bucket policy in your target S3 bucket:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "DelegateS3Access",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::MAIN_ACCOUNT_ID:USER_NAME"
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::BUCKET_NAME/*",
"arn:aws:s3:::BUCKET_NAME"
]
}
]
}
And allow S3 permissions to this user.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Example",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"*"
]
}
]
}
In this case you do not need assume a role on the target account. The user itself will be able to access to bucket in another account.