CloudFormation Elasticsearch Service - Circular dependency between resources on same resource - amazon-web-services

thanks in advance!
I've been stuck on this issue for ages and can't find the solution...
Basically I want to implement the same access policy on my elasticsearch service but when I try to re-create this in cloudformation I receive a circular dependency error.. I know whats causing the error the Fn::GetAtt's which reference the elastic search DomainArn.
So my question is how do I go about implementing this statement without having to reference my elk domain arn?
Template contains errors.: Circular dependency between resources: [XXXXXX]
"XXXXXX": {
"Type": "AWS::Elasticsearch::Domain",
"Properties": {
"AccessPolicies": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": {
"Fn::GetAtt": ["myuser", "Arn"]
}
},
"Action": "es:*",
"Resource": {
"Fn::GetAtt": ["XXXXXX", "DomainArn"]
}
},
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": {
"Fn::GetAtt": ["XXXXXX", "DomainArn"]
},
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"xx.xx.xx.xx",
"xx.xx.xx.xx"
]
}
}
}
]
},
"DomainName": "XXXXXX",
"EBSOptions": {
"EBSEnabled": "True",
"VolumeSize": 10,
"VolumeType": "gp2"
},
"ElasticsearchClusterConfig": {
"InstanceCount": 1,
"InstanceType": "t2.small.elasticsearch"
},
"ElasticsearchVersion": "5.1",
"SnapshotOptions": {
"AutomatedSnapshotStartHour": 0
},
"Tags": {
"Key": "name",
"Value": "XXXXXX"
}
}
},

Rather than use Fn::GetAtt to retrieve the domain ARN, use Fn:Sub to construct the ARN using the rules here (scroll down to "Use the following syntax to specify domain resources for Amazon ES").
{ "Fn::Sub":"arn:aws:es:${AWS::Region}:${AWS::AccountId}:domain/XXXXXX" }

Related

How to deal with multiple duplicate keys (Fn::Sub) in a aws cloudformation template?

I have a policy that is being made in a cloudformation template. I want to add two resources to the policy, they end up being arn::bucket and arn::bucket/*. The issue is that the arn is a parameter and I get the error: [cfn-lint] E0000: Duplicate resource found "Fn::Sub" (line 161). I understand that it doesn't like the duplicates.
"RolePolicies": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "GetGEBucketPutCustomerBucket",
"PolicyDocument": {
"Statement": [
{
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:GetObjectAttributes",
"s3:GetObjectTagging",
"s3:ListBucket",
"s3:DeleteObject"
],
"Effect": "Allow",
"Resource": {
"Fn::Sub": [
"${arn}/*",
{
"arn": {
"Ref": "CustomerS3BucketARN"
}
}
],
"Fn::Sub": [
"${arn}",
{
"arn": {
"Ref": "CustomerS3BucketARN"
}
}
]
}
}
]
},
"Roles": [
{
"Ref": "InstanceRole"
}
]
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "a713fcc6-95c8-423f-a5b8-0020a81e5ce4"
}
}
}
However, this cloudformation is allowed to run, but produces errors. When viewing the policy in IAM console window after create, I see that both of the resources were not created.
IAM policy editor gives me this error.
Ln 1, Col 0Missing Version: We recommend that you specify the Version element to help you with debugging permission issues. since the resource than ends with /* wasn't created by cloud formation.
The easiest way I found to fix this was split up the actions into multiple parts. This allowed me to use the Fn::Sub function and keep things separated. The only downside is that I had to create 4 policy statements instead of 2.
"RolePolicies": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "GetGEBucketPutCustomerBucket",
"PolicyDocument": {
"Statement": [
{
"Action": [
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": {
"Fn::Sub": [
"${arn}",
{
"arn": {
"Ref": "GeS3BucketARN"
}
}
]
}
},
{
"Action": [
"s3:GetObject",
"s3:GetObjectAttributes",
"s3:GetObjectTagging"
],
"Effect": "Allow",
"Resource": {
"Fn::Sub": [
"${arn}/*",
{
"arn": {
"Ref": "GeS3BucketARN"
}
}
]
}
},
{
"Action": [
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": {
"Fn::Sub": [
"${arn}",
{
"arn": {
"Ref": "CustomerS3BucketARN"
}
}
]
}
},
{
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:GetObjectAttributes",
"s3:GetObjectTagging",
"s3:DeleteObject"
],
"Effect": "Allow",
"Resource": {
"Fn::Sub": [
"${arn}/*",
{
"arn": {
"Ref": "CustomerS3BucketARN"
}
}
]
}
}
]
},
"Roles": [
{
"Ref": "InstanceRole"
}
]
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "a713fcc6-95c8-423f-a5b8-0020a81e5ce4"
}
}
}

User: ARN is not authorized to perform: SNS:Publish on resource: ARN (AWS Integration error)

I am trying to monitor an AWS Cloud environment using AquaSec. AquaSec helps you connect with your AWS environment by providing a CloudFormation template, which you would deploy at your AWS environment. Once it is deployed, you use it's ARN in AquaSec to connect/integrate both. Any alert in Aquasec would be sent to an SNS Topic which would send it further to a HTTPS endpoint.
This is the CloudFormation template file,
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Aqua CSPM security scanner cross-account role",
"Parameters": {
"ExternalId": {
"Type": "String",
"Description": "The external ID auto-generated from the Aqua Cloud dashboard. Do not change this value.",
"AllowedPattern": "[-a-z0-9]*",
"ConstraintDescription": "Must be lowercase or numbers, no spaces, dashes ok."
}
},
"Resources": {
"AquaCSPMRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::057012691312:role/lambda-cloudsploit-api"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": {
"Ref": "ExternalId"
}
},
"IpAddress": {
"aws:SourceIp": "3.231.74.65/32"
}
}
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::057012691312:role/lambda-cloudsploit-collector"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": {
"Ref": "ExternalId"
}
},
"IpAddress": {
"aws:SourceIp": "3.231.74.65/32"
}
}
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::057012691312:role/lambda-cloudsploit-remediator"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": {
"Ref": "ExternalId"
}
},
"IpAddress": {
"aws:SourceIp": "3.231.74.65/32"
}
}
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::057012691312:role/lambda-cloudsploit-tasks"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": {
"Ref": "ExternalId"
}
},
"IpAddress": {
"aws:SourceIp": "3.231.74.65/32"
}
}
}
]
},
"Policies": [
{
"PolicyName": "aqua-cspm-supplemental-policy",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ses:DescribeActiveReceiptRuleSet",
"athena:GetWorkGroup",
"logs:DescribeLogGroups",
"logs:DescribeMetricFilters",
"elastictranscoder:ListPipelines",
"elasticfilesystem:DescribeFileSystems",
"servicequotas:ListServiceQuotas",
"ssm:ListAssociations",
"dlm:GetLifecyclePolicies",
"airflow:ListEnvironments",
"glue:GetSecurityConfigurations",
"devops-guru:ListNotificationChannels"
],
"Resource": "*"
}
]
}
}
],
"ManagedPolicyArns": [
"arn:aws:iam::aws:policy/SecurityAudit"
]
}
}
},
"Outputs": {
"AquaCSPMeArn": {
"Description": "The role ARN of the cross-account user. Copy this into Aqua Cloud.",
"Value": {
"Fn::GetAtt": [
"AquaCSPMRole",
"Arn"
]
}
},
"StackVersion": {
"Description": "The Aqua CSPM stack version.",
"Value": "2.0"
}
}
}
I am trying to setup an Amazon SNS Topic, use its ARN here at Aquasec to send out alerts. Once i create an SNS topic, and copy it's ARN at Aquasec, and try out a test notification - I keep getting the error,
{
"message": "User: arn:aws:sts::057012691312:assumed-role/lambda-cloudsploit-api/cloudsploit-api is not authorized to perform: SNS:Publish on resource: arn:aws:sns:us-east-1:940386435759:Sample_Aqua_Integration",
"code": "AuthorizationError",
"time": "2021-04-19T22:15:54.463Z",
"requestId": "bc808944-3430-5683-aed1-d1bc376a70f5",
"statusCode": 403,
"retryable": false,
"retryDelay": 50.2772842473471
}
I have tried nearly every possible way - changed topic policy(various combinations of "Principle" field) in the SNS Topic, tried to give permissions in the specific IAM role. Nothing seems to work and I get the same error.
I feel it has to do something with the template file (in the 'assumeRole' configuration) ?
Any suggestions on how and what to change/try?
Thanks

Planning to create a CFT which need 3 roles with managed policy and a inline policy attached to them

I am trying to create an CFT which has
1. 3 different roles with Managed policy
2. INline policy which should be added to three roles which are created in CFT.
But i could not do as this is throwing me error saying atleast one resource must be defined.
Please help me in achieving this.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"EMRDefaultRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"RoleName": "EMR_DefaultRole",
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {
"Service": "elasticmapreduce.amazonaws.com"
},
"Action": "sts:AssumeRole"
}]
},
"ManagedPolicyArns": [
"arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceRole"
]
}
},
"EMREC2DefaultRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"RoleName": "EMR_EC2_DefaultRole",
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}]
},
"ManagedPolicyArns": [
"arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceforEC2Role"
]
}
},
"EMRAutoScalingDefaultRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"RoleName": "EMR_AutoScaling_DefaultRole",
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {
"Service": [ "elasticmapreduce.amazonaws.com",
"application-autoscaling.amazonaws.com"]
},
"Action": "sts:AssumeRole"
}]
},
"ManagedPolicyArns": [
"arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceforAutoScalingRole"
]
}
},
"EMRS3Policies": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "Moodys-IAM-EMR-S3-Access-Policy",
"PolicyDocument": {
"Statement": [{
"Effect": "Allow",
"Action": [
"s3:HeadBucket",
"s3:GetObject"
],
"Resource": {
"Fn::Join": ["", ["arn:aws:s3:::mit-", {
"Ref": "AWS::AccountId"
}, "-emr-files/*"]]
}
}
]
},
"Roles": [{
"Ref": "EMRDefaultRole"},
{"Ref": "EMREC2DefaultRole"},
{"Ref": "EMRAutoScalingDefaultRole"
}]
}
}
}
}
Like i am expecting three Role where an managed policy and inline policy are attached.
You are missing resource attribute in your role statements.
"Statement": [{
"Effect": "Allow",
"Principal": {
"Service": [ "elasticmapreduce.amazonaws.com",
"application-autoscaling.amazonaws.com"]
},
"Action": "sts:AssumeRole"
}]
This should be (it holds for all of the statements)
"Statement": [{
"Effect": "Allow",
"Principal": {
"Service": [ "elasticmapreduce.amazonaws.com",
"application-autoscaling.amazonaws.com"]
},
"Action": "sts:AssumeRole",
"Resource": [
"arn-of-your-resource-or-wildcard"
]
}]

Is this the correct way to write a CFT for lambda to be triggered by SQS?

Hello I recently made a CFT for my lambda and sqs (both on the same account/region) but I noticed that when I go to the aws console, it shows the SQS resource both on the "input side" and the "output side" of my lambda. Is this intentional? or did I just do my CFTs incorrectly. Below is the relevant JSON code.
"MyLambdaFunctionRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"lambda.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Path": "/",
"Policies": [
{
"PolicyName": "root",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sqs:ReceiveMessage",
"sqs:DeleteMessage",
"sqs:GetQueueAttributes"
],
"Resource": {
"Fn::Sub": "arn:aws:sqs:${AWS::Region}:${AWS::AccountId}:test-${Environment}-my-queue"
}
}
]
}
}
]
}
},
"OutputRouterEventSource": {
"Type": "AWS::Lambda::EventSourceMapping",
"Properties": {
"BatchSize": 10,
"Enabled" : true,
"EventSourceArn": {
"Fn::Sub": "arn:aws:sqs:${AWS::Region}:${AWS::AccountId}:test-${Environment}-my-queue"
},
"FunctionName": {
"Fn::GetAtt": [
"MyLambdaFunction",
"Arn"
]
}
}
},
"OutputRouterLambdaInvokePermission": {
"Type": "AWS::Lambda::Permission",
"Properties": {
"Action": "lambda:InvokeFunction",
"Principal": "sns.amazonaws.com",
"SourceArn": {
"Fn::Sub": "arn:aws:sqs:${AWS::Region}:${AWS::AccountId}:test-${Environment}-my-queue"
},
"FunctionName": {
"Fn::Sub": "test-${Environment}-my-lambda"
}
}
}
It is normal to have SQS on both side in this case: SQS triggers your Lambda function (left side) and your Lambda function's role must have the permission to receive and delete messages from the queue (right side).
(I think your are mixing services in your OutputRouterLambdaInvokePermission resource where the principal is SNS and the source references the arn of a SQS queue, you can drop this resource as it is not needed for SQS triggers)

Incorrect S3 bucket policy is detected for bucket in CloudFormation

I have issues implementing CloudTrail via Cloudformation, with a Incorrect S3 bucket policy is detected for bucket error being thrown when I try to launch the model.
Here is the configuration from the BucketPolicy:
"LogBucketPolicy": {
"Type": "AWS::S3::BucketPolicy",
"Properties": {
"Bucket": {
"Ref": "LogBucket"
},
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSCloudTrailAclCheck",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:s3:::",
{
"Ref": "LogBucket"
}
]
]
}
},
{
"Sid": "AWSCloudTrailWrite",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:s3:::",
{
"Ref": "LogBucket"
},
"/AWSLogs/139339407673/*"
]
]
},
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
}
]
}
}
}
I have copied the template from AWS examples, but let me know if I did a mistake in the implementation.
Edit: The error is not thrown by the bucket policy, but by CloudTrail. Here is the configuration of the bucket:
"Trail": {
"Type": "AWS::CloudTrail::Trail",
"Properties": {
"SnsTopicName": {
"Fn::GetAtt": [
"Topic",
"TopicName"
]
},
"IsLogging": true,
"S3BucketName": {
"Ref": "LogBucket"
}
},
"DependsOn": [
"LogBucket"
]
}
As Krishna has mentioned, the error came from the fact that I didn't put the dependence of the BucketPolicy. When this was done, the stack was deployed with no issues.
I modified your code and it seems to be working for me. Can you try this?
{
"Parameters": {
"LogBucket": {
"Description": "Name Bucket.",
"Type": "String"
}
},
"Resources": {
"LogBucketPolicy": {
"Type": "AWS::S3::BucketPolicy",
"Properties": {
"Bucket": {
"Ref": "LogBucket"
},
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSCloudTrailAclCheck",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:s3:::",
{
"Ref": "LogBucket"
}
]
]
}
},
{
"Sid": "AWSCloudTrailWrite",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:s3:::",
{
"Ref": "LogBucket"
},
"/AWSLogs/139339407673/*"
]
]
},
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
}
]
}
}
}
}
}
Beside the dependency issue mentioned in the accepted answer, the error can also comes from different cases of wrong configuration of the S3 policy.
For example, if we look at the following policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSCloudTrailAclCheck20131101",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": "arn:aws:s3:::myBucketName"
},
{
"Sid": "AWSCloudTrailWrite20131101",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::myBucketName/[optional] myLogFilePrefix/AWSLogs/<account-id>/*"
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
}
]
}
Looking at the Resource block of the 2nd statement:
"Resource": "arn:aws:s3:::myBucketName/[optional] myLogFilePrefix/AWSLogs/<account-id>/*"
Passing wrong values to the Resource block like bad prefix (my case) or forgetting the "*" postfix (like mentioned here in last scenario) can lead to the error.
(*) Example taken from here.