I am trying to deploy a template through CloudFormation. AWS Config with a S3 bucket but I am getting an incorrect policy, unable to write to bucket error.
does anyone know the proper template and code for this action I am trying to complete?
Please help
Here is my code. I am getting incorrect bucket policy can't write to bucket error.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Template for a startup company looking to move their services to the cloud",
"Resources": {
"ResumeConfigRecorder": {
"Type": "AWS::Config::ConfigurationRecorder",
"Properties": {
"Name": "ResumeConfigRecorder",
"RecordingGroup": {
"AllSupported": true
},
"RoleARN": "arn:aws:iam::451750859333:role/aws-service-role/config.amazonaws.com/AWSServiceRoleForConfig"
}
},
"ResumeConfigDeliveryChannel": {
"Type": "AWS::Config::DeliveryChannel",
"Properties": {
"ConfigSnapshotDeliveryProperties": {
"DeliveryFrequency": "Three_Hours"
},
"Name": "ResumeConfigDeliveryChannel",
"S3BucketName": "config-resumematch",
"S3KmsKeyArn": {
"Fn::GetAtt": [
"ConfigKey",
"Arn"
]
}
}
},
"ConfigBucket": {
"Type": "AWS::S3::Bucket",
"Properties": {
"AccessControl": "Private",
"BucketName": "config-resumematch",
"BucketEncryption": {
"ServerSideEncryptionConfiguration": [
{
"BucketKeyEnabled": true,
"ServerSideEncryptionByDefault": {
"KMSMasterKeyID": {
"Ref": "ConfigKey"
},
"SSEAlgorithm": "aws:kms"
}
}
]
},
"Tags": [
{
"Key": "Name",
"Value": "ConfigBucket"
}
]
}
},
"ConfigBucketPolicy": {
"Type": "AWS::S3::BucketPolicy",
"Properties": {
"Bucket": {
"Ref": "ConfigBucket"
},
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSConfigBucketPermissionsCheck",
"Effect": "Allow",
"Principal": {
"Service": "config.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": "arn:aws:s3:::config-resumematch",
"Condition": {
"StringEquals": {
"AWS:SourceAccount": "451750859333"
}
}
},
{
"Sid": "AWSConfigBucketExistenceCheck",
"Effect": "Allow",
"Principal": {
"Service": "config.amazonaws.com"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::config-resumematch",
"Condition": {
"StringEquals": {
"AWS:SourceAccount": "451750859333"
}
}
},
{
"Sid": "AWSConfigBucketDelivery",
"Effect": "Allow",
"Principal": {
"Service": "config.amazonaws.com"
},
"Action": [
"s3:PutObject*"
],
"Resource": "arn:aws:s3:::config-resumematch/AWSLogs/451750859333/Config/*",
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control",
"AWS:SourceAccount": "451750859333"
}
}
}
]
}
}
},
"ConfigKey": {
"Type": "AWS::KMS::Key",
"Properties": {
"Description": "Key to encrypt config records in S3",
"Enabled": true,
"KeyPolicy": {
"Version": "2012-10-17",
"Id": "config-key-1",
"Statement": [
{
"Sid": "Enable IAM Permissions",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::451750859333:root"
},
"Action": "kms:*",
"Resource": "*"
},
{
"Sid": "Allow administration of the key",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::451750859333:user/ecargle"
},
"Action": [
"kms:Put*",
"kms:Create*",
"kms:Describe*",
"kms:Enable*",
"kms:List*",
"kms:Get*",
"kms:Delete*"
],
"Resource": "*"
},
{
"Sid": "Allow config to use KMS key",
"Effect": "Allow",
"Principal": {
"Service": "config.amazonaws.com"
},
"Action": [
"kms:Encrypt*",
"kms:Decrypt*"
],
"Resource": "*"
}
]
},
"KeySpec": "SYMMETRIC_DEFAULT",
"KeyUsage": "ENCRYPT_DECRYPT",
"Tags": [
{
"Key": "Name",
"Value": "ConfigKey"
}
]
}
},
"ConfigKeyAlias": {
"Type": "AWS::KMS::Alias",
"Properties": {
"AliasName": "alias/configKey",
"TargetKeyId": {
"Ref": "ConfigKey"
}
}
},
"ResumeConfigRuleEC2": {
"DependsOn" : "ResumeConfigRecorder",
"Type": "AWS::Config::ConfigRule",
"Properties": {
"ConfigRuleName": "ResumeMatchConfigRuleEC2",
"Description": "rule to monitor the configuration of AWS resources",
"Scope": {
"ComplianceResourceTypes": [
"AWS::EC2::Volume"
]
},
"Source": {
"Owner": "AWS",
"SourceIdentifier": "EC2_EBS_ENCRYPTION_BY_DEFAULT"
}
}
},
"ResumeConfigRuleS3": {
"DependsOn" : "ResumeConfigRecorder",
"Type": "AWS::Config::ConfigRule",
"Properties": {
"ConfigRuleName": "ResumeMatchConfigRuleS3",
"Description": "rule to monitor the configuration of AWS resources",
"Scope": {
"ComplianceResourceTypes": [
"AWS::S3::Bucket"
]
},
"Source": {
"Owner": "AWS",
"SourceIdentifier": "ELB_LOGGING_ENABLED"
}
}
},
"ResumeConfigRuleELB": {
"DependsOn" : "ResumeConfigRecorder",
"Type": "AWS::Config::ConfigRule",
"Properties": {
"ConfigRuleName": "ResumeMatchConfigRuleELB",
"Description": "rule to monitor the configuration of AWS resources",
"Scope": {
"ComplianceResourceTypes": [
"AWS::ElasticLoadBalancingV2::LoadBalancer"
]
},
"Source": {
"Owner": "AWS",
"SourceIdentifier": "ELB_LOGGING_ENABLED"
}
}
}
}
}
Related
I have a policy that is being made in a cloudformation template. I want to add two resources to the policy, they end up being arn::bucket and arn::bucket/*. The issue is that the arn is a parameter and I get the error: [cfn-lint] E0000: Duplicate resource found "Fn::Sub" (line 161). I understand that it doesn't like the duplicates.
"RolePolicies": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "GetGEBucketPutCustomerBucket",
"PolicyDocument": {
"Statement": [
{
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:GetObjectAttributes",
"s3:GetObjectTagging",
"s3:ListBucket",
"s3:DeleteObject"
],
"Effect": "Allow",
"Resource": {
"Fn::Sub": [
"${arn}/*",
{
"arn": {
"Ref": "CustomerS3BucketARN"
}
}
],
"Fn::Sub": [
"${arn}",
{
"arn": {
"Ref": "CustomerS3BucketARN"
}
}
]
}
}
]
},
"Roles": [
{
"Ref": "InstanceRole"
}
]
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "a713fcc6-95c8-423f-a5b8-0020a81e5ce4"
}
}
}
However, this cloudformation is allowed to run, but produces errors. When viewing the policy in IAM console window after create, I see that both of the resources were not created.
IAM policy editor gives me this error.
Ln 1, Col 0Missing Version: We recommend that you specify the Version element to help you with debugging permission issues. since the resource than ends with /* wasn't created by cloud formation.
The easiest way I found to fix this was split up the actions into multiple parts. This allowed me to use the Fn::Sub function and keep things separated. The only downside is that I had to create 4 policy statements instead of 2.
"RolePolicies": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "GetGEBucketPutCustomerBucket",
"PolicyDocument": {
"Statement": [
{
"Action": [
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": {
"Fn::Sub": [
"${arn}",
{
"arn": {
"Ref": "GeS3BucketARN"
}
}
]
}
},
{
"Action": [
"s3:GetObject",
"s3:GetObjectAttributes",
"s3:GetObjectTagging"
],
"Effect": "Allow",
"Resource": {
"Fn::Sub": [
"${arn}/*",
{
"arn": {
"Ref": "GeS3BucketARN"
}
}
]
}
},
{
"Action": [
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": {
"Fn::Sub": [
"${arn}",
{
"arn": {
"Ref": "CustomerS3BucketARN"
}
}
]
}
},
{
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:GetObjectAttributes",
"s3:GetObjectTagging",
"s3:DeleteObject"
],
"Effect": "Allow",
"Resource": {
"Fn::Sub": [
"${arn}/*",
{
"arn": {
"Ref": "CustomerS3BucketARN"
}
}
]
}
}
]
},
"Roles": [
{
"Ref": "InstanceRole"
}
]
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "a713fcc6-95c8-423f-a5b8-0020a81e5ce4"
}
}
}
I am trying to create an AWS CodePipeline that deploys code stored in a CodeCommit repository stored in Account B = HUB Account into Account A = production Account. CodePipeline is available on Account A.
What has been done:
KMS key - Account A:
{
"Version": "2012-10-17",
"Id": "key-consolepolicy-3",
"Statement": [
{
"Sid": "Enable IAM User Permissions",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNT_A:root"
},
"Action": "kms:*",
"Resource": "*"
},
{
"Sid": "Allow access for Key Administrators",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::ACCOUNT_A:role/Admin",
]
},
"Action": [
"kms:Create*",
"kms:Describe*",
"kms:Enable*",
"kms:List*",
"kms:Put*",
"kms:Update*",
"kms:Revoke*",
"kms:Disable*",
"kms:Get*",
"kms:Delete*",
"kms:TagResource",
"kms:UntagResource",
"kms:ScheduleKeyDeletion",
"kms:CancelKeyDeletion"
],
"Resource": "*"
},
{
"Sid": "Allow use of the key",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::ACCOUNT_A:role/service-role/code-build-glue-accountA-role",
"arn:aws:iam::ACCOUNT_A:role/service-role/codepipeline-accountA-service-role",
"arn:aws:iam::ACCOUNT_A:role/cloud-formation-role-accountA-role",
"arn:aws:iam::ACCOUNT_A:role/service-role/code-build-glue-accountA-role",
"arn:aws:iam::ACCOUNT_B:root",
"arn:aws:iam::ACCOUNT_B:role/cloud-formation-accountB-role"
]
},
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": "*"
},
{
"Sid": "Allow attachment of persistent resources",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::ACCOUNT_A:role/service-role/code-build-glue-accountA-role",
"arn:aws:iam::ACCOUNT_A:role/service-role/codepipeline-accountA-service-role",
"arn:aws:iam::ACCOUNT_A:role/cloud-formation-role-accountA-role",
"arn:aws:iam::ACCOUNT_A:role/service-role/code-build-glue-accountA-role",
"arn:aws:iam::ACCOUNT_B:root",
"arn:aws:iam::ACCOUNT_B:role/cloud-formation-accountB-role"
]
},
"Action": [
"kms:CreateGrant",
"kms:ListGrants",
"kms:RevokeGrant"
],
"Resource": "*",
"Condition": {
"Bool": {
"kms:GrantIsForAWSResource": "true"
}
}
}
]}
S3 Bucket policy - Account A
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::ACCOUNT_A:role/codepipeline-service-role",
"arn:aws:iam::ACCOUNT_B:role/cloud-formation-role",
"arn:aws:iam::ACCOUNT_B:role/cross-account-role"
]
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::codepipeline",
"arn:aws:s3:::codepipeline*"
]
}
]}
CodePipeline ServiceRole - policy to assume Account B in Account A:
{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": [
"arn:aws:iam::ACCOUNT_B:role/cross-account-role"
]
}
}
Cross Account Role - trusted relationship - Account B
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNT_A:root"
},
"Action": "sts:AssumeRole",
"Condition": {}
}
]
}
Cross Account policy - Account B
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"codecommit:*",
"codedeploy:*",
"cloudformation:*",
"codebuild:*",
"s3:*",
"iam:PassRole"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"kms:DescribeKey",
"kms:GenerateDataKey*",
"kms:Encrypt",
"kms:ReEncrypt*",
"kms:Decrypt"
],
"Resource": "*"
}
]
}
Pipeline.json file - Account A:
Info: the pipeline has for build and deploy stages two actions: build lambda, build glue and corresponding deploy lambda, deploy glue.
{
"pipeline": {
"name": "cross-account-deployment-code-pipeline",
"roleArn": "arn:aws:iam::ACCOUNT_A:role/service-role/codepipeline-service-role",
"artifactStore": {
"type": "S3",
"location": "codepipeline-eu-west-2",
"encryptionKey": {
"id": "arn:aws:kms:eu-west-2:ACCOUNT_A:key/keyid",
"type": "KMS"
}
},
"stages": [
{
"name": "Source",
"actions": [
{
"name": "Source",
"actionTypeId": {
"category": "Source",
"owner": "AWS",
"provider": "CodeCommit",
"version": "1"
},
"runOrder": 1,
"roleArn": "arn:aws:iam::ACCOUNT_B:role/CrossAccountRole",
"configuration": {
"BranchName": "main",
"OutputArtifactFormat": "CODE_ZIP",
"PollForSourceChanges": "false",
"RepositoryName": "repository-AccountB"
},
"outputArtifacts": [
{
"name": "SourceArtifact"
}
],
"inputArtifacts": [],
"region": "eu-west-2",
"namespace": "SourceVariables"
}
]
},
{
"name": "Archive",
"actions": [
{
"name": "Archive",
"actionTypeId": {
"category": "Deploy",
"owner": "AWS",
"provider": "S3",
"version": "1"
},
"runOrder": 1,
"configuration": {
"BucketName": "assets-codecommit",
"Extract": "true"
},
"outputArtifacts": [],
"inputArtifacts": [
{
"name": "SourceArtifact"
}
],
"region": "eu-west-2"
}
]
},
{
"name": "Build",
"actions": [
{
"name": "Build-Glue-Jobs",
"actionTypeId": {
"category": "Build",
"owner": "AWS",
"provider": "CodeBuild",
"version": "1"
},
"runOrder": 1,
"configuration": {
"ProjectName": "code-build-glue-project"
},
"outputArtifacts": [
{
"name": "BuildArtifact"
}
],
"inputArtifacts": [
{
"name": "SourceArtifact"
}
],
"region": "eu-west-2",
"namespace": "BuildVariables"
},
{
"name": "Build-Lambda",
"actionTypeId": {
"category": "Build",
"owner": "AWS",
"provider": "CodeBuild",
"version": "1"
},
"runOrder": 1,
"configuration": {
"ProjectName": "code-build-lambda-project"
},
"outputArtifacts": [
{
"name": "BuildLambdaArtifact"
}
],
"inputArtifacts": [
{
"name": "SourceArtifact"
}
],
"region": "eu-west-2",
"namespace": "BuildLambdaVariables"
}
]
},
{
"name": "Deploy",
"actions": [
{
"name": "Deploy-Glue-Jobs",
"actionTypeId": {
"category": "Deploy",
"owner": "AWS",
"provider": "CloudFormation",
"version": "1"
},
"runOrder": 1,
"configuration": {
"ActionMode": "REPLACE_ON_FAILURE",
"Capabilities": "CAPABILITY_IAM,CAPABILITY_AUTO_EXPAND",
"OutputFileName": "create-glue-stack-output.json",
"ParameterOverrides": "{\"Environments\":\"PROD\"}",
"RoleArn": "arn:aws:iam::ACCOUNT_B:role/cloudformation-accountB-role",
"StackName": "glue-stack",
"TemplatePath": "BuildArtifact::output-glue-deploy.yaml"
},
"roleArn": "arn:aws:iam::ACCOUNT_B:role/CrossAccountRole",
"outputArtifacts": [],
"inputArtifacts": [
{
"name": "BuildArtifact"
}
],
"region": "eu-west-2",
"namespace": "DeployVariables"
},
{
"name": "Deploy-Lambda-Functions",
"actionTypeId": {
"category": "Deploy",
"owner": "AWS",
"provider": "CloudFormation",
"version": "1"
},
"runOrder": 1,
"configuration": {
"ActionMode": "REPLACE_ON_FAILURE",
"Capabilities": "CAPABILITY_IAM,CAPABILITY_AUTO_EXPAND",
"OutputFileName": "create-lambda-stack-output.json",
"ParameterOverrides": "{\"Environments\":\"PROD\"}",
"RoleArn": "arn:aws:iam::ACCOUNT_B:role/cloudformation-accountB-role",
"StackName": "lambda-stack",
"TemplatePath": "BuildLambdaArtifact::output-lambda-deploy.yaml"
},
"roleArn": "arn:aws:iam::ACCOUNT_B:role/CrossAccountRole",
"outputArtifacts": [],
"inputArtifacts": [
{
"name": "BuildLambdaArtifact"
}
],
"region": "eu-west-2",
"namespace": "DeployLambdaVariables"
}
]
}
],
"version": 62
}
}
When I release the changes in the pipeline - I get error below:
Due to the fact that Cross-account pass role is not allowed.
What I am doing wrong?
I have checked multiple resources and I can not find what I am missing.
I am new to AWS CloudFormation, and I am trying to capture events from SQS Queue and place them in S3 bucket via AWS lambda. Flow of events is
SNS --> SQS <-- Lambda ---> S3 bucket.
I am trying to achieve above flow using cloudFormation template.I am getting below error message after deploying my CloudFormation template. Any help you can provide would be greatly appreciated. Thank you
11:51:56 2022-01-13 17:51:47,930 - INFO - ...
11:52:53 2022-01-13 17:52:48,511 - ERROR - Stack myDemoApp shows a rollback status ROLLBACK_IN_PROGRESS.
11:52:53 2022-01-13 17:52:48,674 - INFO - The following root cause failure event was found in the myDemoApp stack for resource EventStreamLambda:
11:52:53 2022-01-13 17:52:48,674 - INFO - Resource handler returned message: "Error occurred while GetObject. S3 Error Code: NoSuchKey.
S3 Error Message: The specified key does not exist. (Service: Lambda, Status Code: 400, Request
ID: 5f2f9882-a863-4a58-90bd-7e0d0dfdf4d5, Extended Request ID: null)" (RequestToken: 0a95acb4-a677-0a2d-d0bc-8b7487a858ad, HandlerErrorCode: InvalidRequest)
11:52:53 2022-01-13 17:52:48,674 - INFO - ..
MY lambda function is :
import json
import logging
import boto3
logger = logging.getLogger()
logger.setLevel(logging.INFO)
logging.basicConfig(level=logging.INFO,
format='%(asctime)s: %(levelname)s: %(message)s')
def lambda_handler(event, context):
logger.info(f"lambda_handler -- event: {json.dumps(event)}")
s3_bucket = boto3.resource("3")
event_message = json.loads(event["Records"][0]["body"])
s3_bucket.put_object(Bucket="S3DeployBucket", key="data.json", Body=json.dumps(event_message))
My complete CloudFormation template is :
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "myDemoApp Resource Stack",
"Mappings": {
},
"Parameters": {
"S3DeployBucket": {
"Default": "myDemoApp-deploy-bucket",
"Description": "Bucket for deployment configs and artifacts for myDemoApp",
"Type": "String"
},
"EnvName": {
"Description": "Platform environment name for myDemoApp",
"Type": "String"
},
"AuditRecordKeyArn": {
"Description": "ARN for audit record key encryption for myDemoApp",
"Type": "String"
},
"ParentVPCStack": {
"Description": "The name of the stack containing the parent VPC for myDemoApp",
"Type": "String"
},
"StackVersion": {
"Description": "The version of this stack of myDemoApp",
"Type": "String"
},
"EventLogFolderName": {
"Type": "String",
"Description": "folder name for the logs for the event stream of myDemoApp",
"Default": "event_log_stream"
},
"EventLogPartitionKeys": {
"Type": "String",
"Description": "The partition keys that audit logs will write to S3. Use Hive-style naming conventions for automatic Athena/Glue comprehension.",
"Default": "year=!{timestamp:yyyy}/month=!{timestamp:MM}/day=!{timestamp:dd}/hour=!{timestamp:HH}"
},
"AppEventSNSTopicArn": {
"Description": "Events SNS Topic of myDemoApp",
"Type": "String"
},
"ReportingEventsRetentionDays": {
"Default": "2192",
"Description": "The number of days to retain a record used for reporting.",
"Type": "String"
}
},
"Resources": {
"AppEventSQSQueue": {
"Type": "AWS::SQS::Queue"
},
"AppEventSnsSubscription": {
"Type": "AWS::SNS::Subscription",
"Properties": {
"TopicArn": {
"Ref": "AppEventSNSTopicArn"
},
"Endpoint": {
"Fn::GetAtt": [
"AppEventSQSQueue",
"Arn"
]
},
"Protocol": "sqs"
}
},
"S3DeployBucket": {
"Type": "AWS::S3::Bucket",
"DeletionPolicy": "Retain",
"UpdateReplacePolicy": "Retain",
"Properties": {
"BucketEncryption": {
"ServerSideEncryptionConfiguration": [
{
"ServerSideEncryptionByDefault": {
"KMSMasterKeyID": {
"Ref": "AuditRecordKeyArn"
},
"SSEAlgorithm": "aws:kms"
}
}
]
},
"VersioningConfiguration": {
"Status": "Enabled"
},
"LifecycleConfiguration": {
"Rules": [
{
"ExpirationInDays": {
"Ref": "ReportingEventsRetentionDays"
},
"Status": "Enabled"
}
]
}
}
},
"EventStreamLogGroup": {
"Type": "AWS::Logs::LogGroup"
},
"EventLogStream": {
"Type": "AWS::Logs::LogStream",
"Properties": {
"LogGroupName": {
"Ref": "EventStreamLogGroup"
}
}
},
"EventStreamSubscriptionRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "sns.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
},
"Policies": [
{
"PolicyName": "SNSSQSAccessPolicy",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": {
"Action": [
"sqs:*"
],
"Effect": "Allow",
"Resource": "*"
}
}
}
]
}
},
"EventDeliveryRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "sqs.amazonaws.com"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": {
"Ref": "AWS::AccountId"
}
}
}
}
]
}
}
},
"EventSqsQueuePolicy": {
"Type": "AWS::SQS::QueuePolicy",
"Properties": {
"PolicyDocument": {
"Version": "2012-10-17",
"Id": "SqsQueuePolicy",
"Statement": [
{
"Sid": "Allow-SNS-SendMessage",
"Effect": "Allow",
"Principal": "*",
"Action": [
"sqs:SendMessage",
"sqs:ReceiveMessage"
],
"Resource": {
"Fn::GetAtt": [
"EventStreamLambda",
"Arn"
]
},
"Condition": {
"ArnEquals": {
"aws:SourceArn": {
"Ref": "EventSNSTopicArn"
}
}
}
}
]
},
"Queues": [
{
"Ref": "EventSNSTopicArn"
}
]
}
},
"EventDeliveryPolicy": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "sqs_delivery_policy",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject"
],
"Resource": [
{
"Fn::GetAtt": [
"S3DeployBucket",
"Arn"
]
},
{
"Fn::Join": [
"",
[
{
"Fn::GetAtt": [
"S3DeployBucket",
"Arn"
]
},
"/*"
]
]
}
]
},
{
"Effect": "Allow",
"Action": [
"logs:PutLogEvents"
],
"Resource": {
"Fn::Sub": "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:${EventStreamLogGroup}:log-stream:${EventLogStreamLogStream}"
}
},
{
"Effect": "Allow",
"Action": [
"kms:Decrypt",
"kms:GenerateDataKey"
],
"Resource": [
{
"Ref": "AuditRecordKeyArn"
}
],
"Condition": {
"StringEquals": {
"kms:ViaService": {
"Fn::Join": [
"",
[
"s3.",
{
"Ref": "AWS::Region"
},
".amazonaws.com"
]
]
}
},
"StringLike": {
"kms:EncryptionContext:aws:s3:arn": {
"Fn::Join": [
"",
[
{
"Fn::GetAtt": [
"S3DeployBucket",
"Arn"
]
},
"/*"
]
]
}
}
}
}
]
},
"Roles": [
{
"Ref": "EventDeliveryRole"
}
]
}
},
"EventStreamLambda": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Handler": "lambda_function.lambda_handler",
"MemorySize": 128,
"Runtime": "python3.8",
"Timeout": 30,
"FunctionName": "sqs_s3_pipeline_job",
"Role": {
"Fn::GetAtt": [
"SQSLambdaExecutionRole",
"Arn"
]
},
"Code": {
"S3Bucket": {
"Ref": "S3DeployBucket"
},
"S3Key": {
"Ref": "S3DeployBucket"
}
},
"TracingConfig": {
"Mode": "Active"
}
}
},
"SQSLambdaExecutionRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"lambda.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Policies": [
{
"PolicyName": "StreamLambdaLogs",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:*"
],
"Resource": "arn:aws:logs:*:*:*"
}
]
}
},
{
"PolicyName": "SQSLambdaPolicy",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sqs:ReceiveMessage",
"sqs:DeleteMessage",
"sqs:GetQueueAttributes",
"sqs:ChangeMessageVisibility"
],
"Resource":"*"
}
]
}
}
]
}
}
},
"Outputs": {
"VpcSubnet3ExportKey": {
"Value": {
"Fn::Sub": "${ParentVPCStack}-privateSubnet3"
}
}
}
}
SubscriptionRoleArn is only for kinesis:
This property applies only to Amazon Kinesis Data Firehose delivery stream subscriptions.
While launching on-demand instance from AWS I'm getting the following error:
An error occurred (UnauthorizedOperation) when calling the RunInstances operation: You are not authorized to perform this operation.
With some Encoded authorization failure message.
But I'm unable to replicate the actual issue from the response as the decoded JSON message has an empty failure object although I'm able to launch spot-instance from the same IAM Policy.
"allowed": false,
"explicitDeny": false,
"matchedStatements": {
"items": []
},
"failures": {
"items": []
},
"context": {
"principal": {
"id": "XXXXXXXXXXXXXXXXXXXX",
"name": "user_name",
"arn": "arn:aws:iam::account_id:user/user_name"
},
"action": "ec2:RunInstances",
"resource": "arn:aws:ec2:us-east-1:account_id:instance/*",
"conditions": {
"items": [
{
"key": "ec2:InstanceMarketType",
"values": {
"items": [
{
"value": "on-demand"
}
]
}
},
{
"key": "aws:Resource",
"values": {
"items": [
{
"value": "instance/*"
}
]
}
},
{
"key": "aws:Account",
"values": {
"items": [
{
"value": "account_id"
}
]
}
},
{
"key": "ec2:AvailabilityZone",
"values": {
"items": [
{
"value": "us-east-1a"
}
]
}
},
{
"key": "ec2:ebsOptimized",
"values": {
"items": [
{
"value": "false"
}
]
}
},
{
"key": "ec2:IsLaunchTemplateResource",
"values": {
"items": [
{
"value": "false"
}
]
}
},
{
"key": "ec2:InstanceType",
"values": {
"items": [
{
"value": "m1.medium"
}
]
}
},
{
"key": "ec2:RootDeviceType",
"values": {
"items": [
{
"value": "ebs"
}
]
}
},
{
"key": "aws:Region",
"values": {
"items": [
{
"value": "us-east-1"
}
]
}
},
{
"key": "aws:Service",
"values": {
"items": [
{
"value": "ec2"
}
]
}
},
{
"key": "ec2:InstanceID",
"values": {
"items": [
{
"value": "*"
}
]
}
},
{
"key": "aws:Type",
"values": {
"items": [
{
"value": "instance"
}
]
}
},
{
"key": "ec2:Tenancy",
"values": {
"items": [
{
"value": "default"
}
]
}
},
{
"key": "ec2:Region",
"values": {
"items": [
{
"value": "us-east-1"
}
]
}
},
{
"key": "aws:ARN",
"values": {
"items": [
{
"value": "arn:aws:ec2:us-east-1:account_id:instance/*"
}
]
}
}
]
}
}
}```
**Below is my IAM Policy**
```{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"ec2:StartInstances",
"ec2:StopInstances"
],
"Resource": "arn:aws:ec2:us-east-1:account_id:instance/m*,t*",
"Condition": {
"StringLike": {
"ec2:InstanceType": [
"m*",
"t*"
]
}
}
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "ec2:RunInstances",
"Resource": [
"arn:aws:ec2:*::image/ami-*",
"arn:aws:ec2:*:*:subnet/*",
"arn:aws:ec2:*:*:key-pair/*",
"arn:aws:ec2:*:*:volume/*",
"arn:aws:ec2:us-east-1:account_id:instance/m*,t*",
"arn:aws:ec2:*:*:security-group/*",
"arn:aws:ec2:*:*:network-interface/*"
],
"Condition": {
"StringLike": {
"ec2:InstanceType": [
"m*",
"t*"
]
}
}
},
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": "ec2:TerminateInstances",
"Resource": "arn:aws:ec2:us-east-1:account_id:instance/*",
"Condition": {
"StringLike": {
"ec2:InstanceType": [
"m*",
"t*"
]
}
}
},
{
"Sid": "VisualEditor3",
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:PurchaseReservedInstancesOffering",
"ec2:DescribeAvailabilityZones",
"ec2:EnableEbsEncryptionByDefault",
"ec2:DescribeReservedInstancesOfferings",
"ec2:DescribeReservedInstances",
"ec2:ModifyReservedInstances"
],
"Resource": "*"
},
{
"Sid": "VisualEditor4",
"Effect": "Allow",
"Action": [
"ec2:ModifyVolumeAttribute",
"ec2:DescribeInstances",
"ec2:GetEbsEncryptionByDefault",
"ec2:ExportClientVpnClientConfiguration",
"ec2:GetHostReservationPurchasePreview",
"ec2:DeleteVolume",
"ec2:GetLaunchTemplateData",
"ec2:SearchTransitGatewayRoutes",
"ec2:DescribeVolumeStatus",
"ec2:DescribeVolumes",
"ec2:GetEbsDefaultKmsKeyId",
"ec2:DetachVolume",
"ec2:ModifyVolume",
"ec2:GetTransitGatewayAttachmentPropagations",
"ec2:GetReservedInstancesExchangeQuote",
"ec2:DescribeVolumeAttribute",
"ec2:CreateVolume",
"ec2:GetPasswordData",
"ec2:GetTransitGatewayRouteTablePropagations",
"ec2:AttachVolume",
"ec2:PurchaseReservedInstancesOffering",
"ec2:RequestSpotInstances",
"ec2:GetCapacityReservationUsage",
"ec2:ExportClientVpnClientCertificateRevocationList",
"ec2:CreateSecurityGroup",
"ec2:GetTransitGatewayRouteTableAssociations",
"ec2:DescribeInstanceStatus",
"ec2:DescribeTags",
"ec2:ImportSnapshot",
"sts:*",
"ec2:Describe*",
"ec2:GetConsole*",
],
"Resource": "*"
},
{
"Sid": "VisualEditor5",
"Effect": "Allow",
"Action": [
"ec2:AuthorizeSecurityGroupEgress",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:DeleteSecurityGroup"
],
"Resource": "*"
},
{
"Sid": "VisualEditor6",
"Effect": "Allow",
"Action": [
"ec2:DeleteTags",
"ec2:CreateTags",
"ec2:GetConsoleScreenshot"
],
"Resource": "*"
}
]
}```
along with IAM all readonly permissions
Here is a policy that grants permission to launch an instance in the M or T family:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "InstanceType",
"Effect": "Allow",
"Action": [
"ec2:TerminateInstances",
"ec2:StartInstances",
"ec2:RunInstances",
"ec2:StopInstances"
],
"Resource": "arn:aws:ec2:*:*:instance/*",
"Condition": {
"StringLike": {
"ec2:InstanceType": [
"t*",
"m*"
]
}
}
},
{
"Sid": "Any",
"Effect": "Allow",
"Action": "ec2:RunInstances",
"Resource": [
"arn:aws:ec2:*:*:subnet/*",
"arn:aws:ec2:*:*:volume/*",
"arn:aws:ec2:*:*:security-group/*",
"arn:aws:ec2:*:*:placement-group/*",
"arn:aws:ec2:*:*:network-interface/*",
"arn:aws:ec2:*::image/*"
]
}
]
}
It is based on the example from Example Policies for Working in the Amazon EC2 Console - Amazon Elastic Compute Cloud.
After a bit of playing around, it seems that the ec2:InstanceType parameter is only relevant for the instance/* resource type.
Here's a sample CLI command that I used to test it:
aws ec2 run-instances --image-id ami-abcd1234 --security-group-id sg-abcd1234 --instance-type t2.nano
Please note that it does not grant permission to add tags or pass an IAM Role in the RunInstances command.
I have issues implementing CloudTrail via Cloudformation, with a Incorrect S3 bucket policy is detected for bucket error being thrown when I try to launch the model.
Here is the configuration from the BucketPolicy:
"LogBucketPolicy": {
"Type": "AWS::S3::BucketPolicy",
"Properties": {
"Bucket": {
"Ref": "LogBucket"
},
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSCloudTrailAclCheck",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:s3:::",
{
"Ref": "LogBucket"
}
]
]
}
},
{
"Sid": "AWSCloudTrailWrite",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:s3:::",
{
"Ref": "LogBucket"
},
"/AWSLogs/139339407673/*"
]
]
},
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
}
]
}
}
}
I have copied the template from AWS examples, but let me know if I did a mistake in the implementation.
Edit: The error is not thrown by the bucket policy, but by CloudTrail. Here is the configuration of the bucket:
"Trail": {
"Type": "AWS::CloudTrail::Trail",
"Properties": {
"SnsTopicName": {
"Fn::GetAtt": [
"Topic",
"TopicName"
]
},
"IsLogging": true,
"S3BucketName": {
"Ref": "LogBucket"
}
},
"DependsOn": [
"LogBucket"
]
}
As Krishna has mentioned, the error came from the fact that I didn't put the dependence of the BucketPolicy. When this was done, the stack was deployed with no issues.
I modified your code and it seems to be working for me. Can you try this?
{
"Parameters": {
"LogBucket": {
"Description": "Name Bucket.",
"Type": "String"
}
},
"Resources": {
"LogBucketPolicy": {
"Type": "AWS::S3::BucketPolicy",
"Properties": {
"Bucket": {
"Ref": "LogBucket"
},
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSCloudTrailAclCheck",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:s3:::",
{
"Ref": "LogBucket"
}
]
]
}
},
{
"Sid": "AWSCloudTrailWrite",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:s3:::",
{
"Ref": "LogBucket"
},
"/AWSLogs/139339407673/*"
]
]
},
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
}
]
}
}
}
}
}
Beside the dependency issue mentioned in the accepted answer, the error can also comes from different cases of wrong configuration of the S3 policy.
For example, if we look at the following policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSCloudTrailAclCheck20131101",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": "arn:aws:s3:::myBucketName"
},
{
"Sid": "AWSCloudTrailWrite20131101",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::myBucketName/[optional] myLogFilePrefix/AWSLogs/<account-id>/*"
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
}
]
}
Looking at the Resource block of the 2nd statement:
"Resource": "arn:aws:s3:::myBucketName/[optional] myLogFilePrefix/AWSLogs/<account-id>/*"
Passing wrong values to the Resource block like bad prefix (my case) or forgetting the "*" postfix (like mentioned here in last scenario) can lead to the error.
(*) Example taken from here.