What principal should be used when specifying a Kinesis stream as a principal in AWS? Specifically I want to write an AWS::Lambda::Permission resource in CloudFormation: the lambda will be triggered by a Kinesis stream. Principal is required from the docs
None of these work for me:
kinesis.amazonaws.com
kinesisstream.amazonaws.com
kinesisstreams.amazonaws.com
kinesis-stream.amazonaws.com
I haven't found any documentation which gives a mapping of service to principal. I found this page, which shows the principal for Kinesis Analytics is kinesisanalytics.amazonaws.com.
I looked at the AWS service namespaces page, but it didn't have any information about principals. The namespace for streams appeared to just be kinesis, but that didn't work for me, as mentioned above.
The correct one is kinesisanalytics.amazonaws.com
Yet let me assume that you want to trigger lambda once a new record putted into kinesis, then all what you need is:
Create IAM Role with the following assume role policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow"
}
]
}
Create an IAM Role Policy "to be attached to the Role created in step 1":
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:{aws_region}:{account_id}:*:*",
"Effect": "Allow"
},
{
"Effect": "Allow",
"Action": [
"kinesis:GetRecords",
"kinesis:GetShardIterator",
"kinesis:DescribeStream",
"kinesis:ListStreams"
],
"Resource": "arn:aws:kinesis:{aws_region}:{account_id}:stream/{kinesis_stream_name}"
}]
}
Attach IAM Role from step 1 to the lambda function
Do lambda event source mapping "Add Trigger" with the following attribute:
batch_size = 100
starting position = "LATEST"
Enabled = True
Related
I have a NodeJS application that publishes message to AWS SNS topic string and a AWS SQS subscription for the same. On the SQS console, I can see the published message. However, I am not clear with the access policy of the SQS queue.
This answer mentions the use of "Principal": "*" - but, that is very broad. One could probably use "Principal" : {"AWS": "*"}; but, that isn't narrow either.
{
"Version": "2012-10-17",
"Id": "Policy1607949016538",
"Statement": [
{
"Sid": "Stmt1607949012567",
"Effect": "Allow",
"Principal": "*",
"Action": [
"sqs:ReceiveMessage",
"sqs:SendMessage"
],
"Resource": "arn:aws:sqs:ap-south-1:463113000000:orders"
}
]
}
Questions
While delivering a message to SQS queue, as a result of subscription, which user is in effect? Same as the one who published to the topic?
I could get the messages to flow into the queue only when I used "Principal" : {"AWS": "*"}. So, how should I define a restrictive policy such that messages are written to queues only as a result of subscription?
What is the equivalent in the AWS SQS CLI to create a queue with "Principal" : {"AWS": "*"} permissions?
The only user that matters is the one that qualifies for the policy as defined for subscription and SQS access policy.
The Condition in policy document can make the overall policy restrictive. See example below.
Adding SQS Permissions with conditions using AWS CLI Command
Example policy document restricting access to account ID.
{
"Version": "2012-10-17",
"Id": "Policy1607960702002",
"Statement": [
{
"Sid": "Stmt1607960701004",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"sqs:ReceiveMessage",
"sqs:SendMessage"
],
"Resource": "arn:aws:sqs:ap-south-1:463113000000:orders",
"Condition": {
"StringEquals": {
"aws:SourceAccount": "463113000000"
}
}
}
]
}
I am implementing CodePipeline; using GitHub, CodeBuild and Amazon ECS (blue/green). The role I am using, is the one generated by the Pipeline: ecsTaskExecutionRole
When generated, it is equipped with the following policies:
AmazonECSTaskExecutionRolePolicy (containing the following actions):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ecr:GetAuthorizationToken",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
"ecr:BatchGetImage",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "*"
}
]}
And the following Trust relationships:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"codebuild.amazonaws.com",
"ecs-tasks.amazonaws.com",
]
},
"Action": "sts:AssumeRole"
}
]
}
Given that the role is auto-generated, one would assume that either it would have ALL the necessary permissions (for the pipeline to function) OR AWS would have a guide on which permissions to assign (to either a policy OR the trust relationship configuration).
Despite, updating the trust relationship to include:
"Service": [
"codebuild.amazonaws.com",
"ecs-tasks.amazonaws.com",
"ec2.amazonaws.com",
"codedeploy.amazonaws.com",
"codepipeline.amazonaws.com",
"s3.amazonaws.com"
]
I still get the error:
I have seen this issue raised in multiple blogs/forum, spanning the past 1-2 years; it's incredible that this is still not properly documented as part of the AWS tutorials (or relative blogs).
"The provided role does not have sufficient permissions to access CodeDeploy"
This error suggests the CodePipeline role is missing "codedeploy:" related permissions.
Can you please add
codedeploy:*
to the role and try again.
If you do not want to add all CodeDeploy permissions, you will need to investigate 'AccessDenied' calls in Cloudtrail and allow just those. Usually these are the required ones:
{
"Action": [
"codedeploy:CreateDeployment",
"codedeploy:GetApplicationRevision",
"codedeploy:GetApplication",
"codedeploy:GetDeployment",
"codedeploy:GetDeploymentConfig",
"codedeploy:RegisterApplicationRevision"
],
"Resource": "*",
"Effect": "Allow"
},
The default "CodePipeline Service Role Policy" is documented here:
[1] Manage the CodePipeline Service Role - Review the Default CodePipeline Service Role Policy - https://docs.aws.amazon.com/codepipeline/latest/userguide/how-to-custom-role.html#view-default-service-role-policy
I am trying to invoke a lambda function from an API Gateway. I have followed the next tutorial: https://docs.aws.amazon.com/apigateway/latest/developerguide/integrating-api-with-aws-services-lambda.html
However, I get the following error when I test it from the web of API Gateway:
Execution failed due to configuration error: API Gateway does not have permission to assume the provided role
I have search in google and I have not been able to solve it (this, for instance).
If I go to the IAM Management Console, I can see that the trust relationship allows API Gateway to assume the rol, and the JSON of the trust relationship is the following:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"apigateway.amazonaws.com",
"lambda.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}
I have tried also with:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": [
"lambda.amazonaws.com",
"apigateway.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}
The policy of the role is the next:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"lambda:InvokeFunction"
],
"Resource": [
"*"
]
}
]
}
What is wrong here? Thank you
To fix this go to the role in your IAM and select the “Trust Relationships” tab. From here edit the policy and for the Principal Service add in “apigateway.amazonaws.com” as seen below. This will grant the API Gateway the ability to assume roles to run your function in addition to the existing lambda permission.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"apigateway.amazonaws.com",
"lambda.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}
I guess you have not attached the role to the invoking method i.e the api gateway
Attaching the created role to the api gateway is needed for api to execute the lamda.
Under Execution role, choose Choose an existing role.
Enter the role ARN for the lambda_invoke_function_assume_apigw_role
role you created earlier.
Choose Save.
AWS Link
I want to set up IAM policies to allow an user to publish to SNS to send SMS and to publish to a specific SNS arn.
I have found a way to allow SMS publish without allowing any SNS publish :
Authorization when sending a text message using AmazonSNSClient
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Deny",
"Action": [
"sns:Publish"
],
"Resource": "arn:aws:sns:*:*:*"
},
{
"Effect": "Allow",
"Action": [
"sns:Publish"
],
"Resource": "*"
}
]
}
But this policy is explicitly denying all other SNS publish, so I can't add a policy allowing a specific SNS.
The problem is that SMS publish does not have a specific arn.
So I am looking at conditions to find a way to limit the allow to publish only SMS. But the specific SMS parameters (PhoneNumber cf https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/SNS.html#publish-property) cannot be filtered in condition :
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "sns:Publish",
"Resource": "*",
"Condition": {"Null":{"PhoneNumber":"false"}}
}
]
}
Is there a way to accomplish such a policy ?
Actually to do the trick I found a way using an allow whit the NotResource JSON Policy Element (spec). I use this property to match the resources which do NOT have an ARN:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sns:Publish"
],
"NotResource": "arn:aws:sns:*:*:*"
}
]
}
With this trick I can allow all sns Publish without ARN (but I don't know if there is any other services then SMS...).
This also allow me to allow specifics ARN in another policy.
When I originally setup CloudWatch, I created an EC2 Instance Profile to automatically grant access to write to the account's own CloudWatch service. Now, I would like to consolidate the logs from several accounts into a central account.
I'd like to implement a simplified architecture that is based on Centralized Logging on AWS. However, these logs will feed an on-premise ELK stack, so I'm only trying to implement the components outlined in red. I would like to solve this without the use of Kinesis.
Either the CloudWatch Agent (CWAgent) doesn't support assuming a role or I can't wrap my mind around how to craft the EC2 Instance Profile to allow the CWAgent to assume a role in a different account.
Logging Target (AWS Account 111111111111)
IAM LogStreamerRole:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::999999999999:role/EC2CloudWatchLoggerRole"
]
},
"Action": "sts:AssumeRole",
"Condition": {}
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": [
"arn:aws:logs:*:*:*"
]
}
]
}
Logging Source (AWS Account 999999999999)
IAM Instance Profile Role:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::111111111111:role/LogStreamerRole"
}
]
}
The CWAgent is producing the following error:
/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log:
2018-02-12T23:27:43Z E! CreateLogStream / CreateLogGroup with log group name Linux/var/log/messages stream name i-123456789abcdef has errors. Will retry the request: AccessDeniedException: User: arn:aws:sts::999999999999:assumed-role/EC2CloudWatchLoggerRole/i-123456789abcdef is not authorized to perform: logs:CreateLogStream on resource: arn:aws:logs:us-west-2:999999999999:log-group:Linux/var/log/messages:log-stream:i-123456789abcdef
status code: 400, request id: 53271811-1234-11e8-afe1-a3c56071215e
It is still trying to write to its own CloudWatch service, instead of to the central CloudWatch service.
From the logs, I see that the instance profile is used.
arn:aws:sts::999999999999:assumed-role/EC2CloudWatchLoggerRole/i-123456789abcdef
Just add the following to the /etc/awslogs/awscli.conf to assume the LogStreamerRole role.
role_arn = arn:aws:iam::111111111111:role/LogStreamerRole
credential_source=Ec2InstanceMetadata