AWS CodeBuild create from CloudFormation can not find source - amazon-web-services

I have am building a simple code AWS Code Build task, loading code from code commit.
When I create the task from the console all is good. But if I create the task from CloudFormation, I get the following error when running the task:
CLIENT_ERROR: repository not found for primary source and source version
My CloudFormation looks like this:
"Project": {
"Type": "AWS::CodeBuild::Project",
"Properties": {
"Name": "DockerBuild",
"Description": "Build a docker Image of the Project",
"ServiceRole": {
"Fn::GetAtt": [
"CodeBuildServiceRole",
"Arn"
]
},
"Artifacts": {
"Type": "no_artifacts"
},
"Environment": {
"Type": "LINUX_CONTAINER",
"ComputeType": "BUILD_GENERAL1_SMALL",
"Image": "aws/codebuild/java:openjdk-8",
"EnvironmentVariables": [
{
"Name": "AWS_DEFAULT_REGION",
"Type": "PLAINTEXT",
"Value": "ca-central-1"
},
{
"Name": "AWS_ACCOUNT_ID",
"Type": "PLAINTEXT",
"Value": {
"Ref": "ServiceName"
}
},
{
"Name": "IMAGE_TAG",
"Type": "PLAINTEXT",
"Value": "latest"
},
{
"Name": "IMAGE_REPO_NAME",
"Type": "PLAINTEXT",
"Value": {"Ref":"Repository"}
}
]
},
"Source": {
"Location": "git-codecommit.ca-central-1.amazonaws.com/v1/repos/demoRepo",
"Type": "CODECOMMIT"
},
"SourceVersion": "refs/heads/main",
"TimeoutInMinutes": 10,
"Tags": []
}
}
If I try to update the Source from the task, it finds all commit number, so the source should be good.
Note that, not sure if it is related or not, but if I try to save my update on source with the option "allow AWS CodeBuild to modify this service role", I get the following error:
The policy was not attached to role CodeBuild-CodeBuildServiceRole-XXXXXXX
I am not sure why or if this has an impact.
If that can help, here is the role created through CloudFormation:
"CodeBuildServiceRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"codebuild.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Path": "/",
"Policies": [
{
"PolicyName": "root",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"codebuild:*",
"codecommit:*",
"cloudwatch:GetMetricStatistics",
"ec2:DescribeVpcs",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ecr:DescribeRepositories",
"ecr:ListImages",
"elasticfilesystem:DescribeFileSystems",
"events:DeleteRule",
"events:DescribeRule",
"events:DisableRule",
"events:EnableRule",
"events:ListTargetsByRule",
"events:ListRuleNamesByTarget",
"events:PutRule",
"events:PutTargets",
"events:RemoveTargets",
"logs:GetLogEvents",
"logs:CreateLogGroup",
"logs:CreateLogStream",
"s3:GetBucketLocation",
"s3:ListAllMyBuckets"
],
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"logs:DeleteLogGroup"
],
"Effect": "Allow",
"Resource": "arn:aws:logs:*:*:log-group:/aws/codebuild/*:log-stream:*"
},
{
"Effect": "Allow",
"Action": [
"ssm:PutParameter"
],
"Resource": "arn:aws:ssm:*:*:parameter/CodeBuild/*"
},
{
"Effect": "Allow",
"Action": [
"ssm:StartSession"
],
"Resource": "arn:aws:ecs:*:*:task/*/*"
},
{
"Sid": "CodeStarConnectionsReadWriteAccess",
"Effect": "Allow",
"Action": [
"codestar-connections:CreateConnection",
"codestar-connections:DeleteConnection",
"codestar-connections:UpdateConnectionInstallation",
"codestar-connections:TagResource",
"codestar-connections:UntagResource",
"codestar-connections:ListConnections",
"codestar-connections:ListInstallationTargets",
"codestar-connections:ListTagsForResource",
"codestar-connections:GetConnection",
"codestar-connections:GetIndividualAccessToken",
"codestar-connections:GetInstallationUrl",
"codestar-connections:PassConnection",
"codestar-connections:StartOAuthHandshake",
"codestar-connections:UseConnection"
],
"Resource": "arn:aws:codestar-connections:*:*:connection/*"
},
{
"Sid": "CodeStarNotificationsReadWriteAccess",
"Effect": "Allow",
"Action": [
"codestar-notifications:CreateNotificationRule",
"codestar-notifications:DescribeNotificationRule",
"codestar-notifications:UpdateNotificationRule",
"codestar-notifications:DeleteNotificationRule",
"codestar-notifications:Subscribe",
"codestar-notifications:Unsubscribe"
],
"Resource": "*",
"Condition": {
"StringLike": {
"codestar-notifications:NotificationsForResource": "arn:aws:codebuild:*"
}
}
},
{
"Sid": "CodeStarNotificationsListAccess",
"Effect": "Allow",
"Action": [
"codestar-notifications:ListNotificationRules",
"codestar-notifications:ListEventTypes",
"codestar-notifications:ListTargets",
"codestar-notifications:ListTagsforResource"
],
"Resource": "*"
},
{
"Sid": "CodeStarNotificationsSNSTopicCreateAccess",
"Effect": "Allow",
"Action": [
"sns:CreateTopic",
"sns:SetTopicAttributes"
],
"Resource": "arn:aws:sns:*:*:codestar-notifications*"
},
{
"Sid": "SNSTopicListAccess",
"Effect": "Allow",
"Action": [
"sns:ListTopics",
"sns:GetTopicAttributes"
],
"Resource": "*"
},
{
"Sid": "CodeStarNotificationsChatbotAccess",
"Effect": "Allow",
"Action": [
"chatbot:DescribeSlackChannelConfigurations"
],
"Resource": "*"
}
]
}
}
]
}
},

So the problem is the Location needs ton include https in front. Value should have been
"Location": "https://git-codecommit.ca-central-1.amazonaws.com/v1/repos/demoRepo"

Related

AWS Lambda does not seem to get an private IP address

I am testing the Vault Lambda Extension which allows to reads secrets from Hashicorp Vault without using tokens.
I therefore created a demo lambda function which use the layer "Vault Lambda Extension".
The lambda crashes before the python code is run.
Example of the execution result:
foo
Response
{
"errorType": "Extension.Crash",
"errorMessage": "RequestId: b16468f9-009c-4b1a-91ca-8b0899e01b7c Error: exit status 1"
}
Function Logs
2023-02-15T18:15:10.999Z [INFO] vault-lambda-extension: Initialising
2023-02-15T18:15:11.006Z [ERROR] Fatal error, exiting: error="error logging in to Vault: read tcp 169.254.76.1:39510->10.73.1.100:8200: read: connection reset by peer"
EXTENSION Name: vault-lambda-extension State: Started Events: []
2023-02-15T18:15:12.712Z [INFO] vault-lambda-extension: Initialising
2023-02-15T18:15:12.773Z [ERROR] Fatal error, exiting: error="error logging in to Vault: read tcp 169.254.76.1:39514->10.73.1.100:8200: read: connection reset by peer"
EXTENSION Name: vault-lambda-extension State: Started Events: []
START RequestId: b16468f9-009c-4b1a-91ca-8b0899e01b7c Version: $LATEST
RequestId: b16468f9-009c-4b1a-91ca-8b0899e01b7c Error: exit status 1
Extension.Crash
END RequestId: b16468f9-009c-4b1a-91ca-8b0899e01b7c
REPORT RequestId: b16468f9-009c-4b1a-91ca-8b0899e01b7c Duration: 1694.95 ms Billed Duration: 1695 ms Memory Size: 128 MB Max Memory Used: 24 MB
This lambda is our Dev VPC private subnet.
We are therefore expecting it would receive a private IP address of this subnet CIDR.
But as you can see the address IP of the lambda is 169.254.76.1 (which is a Link-local address, an address used by Linux when it is not able to set an private IP address ).
The role (modified) set for this lambda is
{
"partial": false,
"policies": [
{
"arn": "arn:aws:iam::123456789012:policy/aws_iam_policy_lambda_vault_lab",
"document": {
"Statement": [
{
"Action": [
"ec2:DescribeNetworkInterfaces",
"ec2:CreateNetworkInterface",
"ec2:DeleteNetworkInterface",
"ec2:DescribeInstances",
"ec2:AttachNetworkInterface",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeVpcs",
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:DescribeLogGroups",
"logs:DescribeLogStreams",
"logs:PutLogEvents",
"logs:GetLogEvents",
"logs:FilterLogEvents"
],
"Effect": "Allow",
"Resource": "*"
}
],
"Version": "2012-10-17"
},
"id": "ANPATRANPATRANPATRANPAT",
"name": "aws_iam_policy_lambda_vault_lab",
"type": "managed"
}
],
"resources": {
"ec2": {
"service": {
"icon": "data:image/svg+xml;base64,PHN2ZyB2a...CAgPC9nPgogIDwvZz4KPC9zdmc+Cg==",
"name": "Amazon EC2"
},
"statements": [
{
"action": "ec2:DescribeNetworkInterfaces",
"effect": "Allow",
"resource": "*",
"service": "ec2",
"source": {
"index": "0",
"policyName": "aws_iam_policy_lambda_vault_lab",
"policyType": "managed"
}
},
{
"action": "ec2:CreateNetworkInterface",
"effect": "Allow",
"resource": "*",
"service": "ec2",
"source": {
"index": "0",
"policyName": "aws_iam_policy_lambda_vault_lab",
"policyType": "managed"
}
},
{
"action": "ec2:DeleteNetworkInterface",
"effect": "Allow",
"resource": "*",
"service": "ec2",
"source": {
"index": "0",
"policyName": "aws_iam_policy_lambda_vault_lab",
"policyType": "managed"
}
},
{
"action": "ec2:DescribeInstances",
"effect": "Allow",
"resource": "*",
"service": "ec2",
"source": {
"index": "0",
"policyName": "aws_iam_policy_lambda_vault_lab",
"policyType": "managed"
}
},
{
"action": "ec2:AttachNetworkInterface",
"effect": "Allow",
"resource": "*",
"service": "ec2",
"source": {
"index": "0",
"policyName": "aws_iam_policy_lambda_vault_lab",
"policyType": "managed"
}
},
{
"action": "ec2:DescribeSecurityGroups",
"effect": "Allow",
"resource": "*",
"service": "ec2",
"source": {
"index": "0",
"policyName": "aws_iam_policy_lambda_vault_lab",
"policyType": "managed"
}
},
{
"action": "ec2:DescribeSubnets",
"effect": "Allow",
"resource": "*",
"service": "ec2",
"source": {
"index": "0",
"policyName": "aws_iam_policy_lambda_vault_lab",
"policyType": "managed"
}
},
{
"action": "ec2:DescribeVpcs",
"effect": "Allow",
"resource": "*",
"service": "ec2",
"source": {
"index": "0",
"policyName": "aws_iam_policy_lambda_vault_lab",
"policyType": "managed"
}
}
]
},
"logs": {
"service": {
"icon": "data:image/svg+xml;base64,PHN2ZyB2aWV3Qm94PSIwIDAg...z4KICA8L2c+Cjwvc3ZnPgo=",
"name": "Amazon CloudWatch Logs"
},
"statements": [
{
"action": "logs:CreateLogGroup",
"effect": "Allow",
"resource": "*",
"service": "logs",
"source": {
"index": "0",
"policyName": "aws_iam_policy_lambda_vault_lab",
"policyType": "managed"
}
},
{
"action": "logs:CreateLogStream",
"effect": "Allow",
"resource": "*",
"service": "logs",
"source": {
"index": "0",
"policyName": "aws_iam_policy_lambda_vault_lab",
"policyType": "managed"
}
},
{
"action": "logs:DescribeLogGroups",
"effect": "Allow",
"resource": "*",
"service": "logs",
"source": {
"index": "0",
"policyName": "aws_iam_policy_lambda_vault_lab",
"policyType": "managed"
}
},
{
"action": "logs:DescribeLogStreams",
"effect": "Allow",
"resource": "*",
"service": "logs",
"source": {
"index": "0",
"policyName": "aws_iam_policy_lambda_vault_lab",
"policyType": "managed"
}
},
{
"action": "logs:PutLogEvents",
"effect": "Allow",
"resource": "*",
"service": "logs",
"source": {
"index": "0",
"policyName": "aws_iam_policy_lambda_vault_lab",
"policyType": "managed"
}
},
{
"action": "logs:GetLogEvents",
"effect": "Allow",
"resource": "*",
"service": "logs",
"source": {
"index": "0",
"policyName": "aws_iam_policy_lambda_vault_lab",
"policyType": "managed"
}
},
{
"action": "logs:FilterLogEvents",
"effect": "Allow",
"resource": "*",
"service": "logs",
"source": {
"index": "0",
"policyName": "aws_iam_policy_lambda_vault_lab",
"policyType": "managed"
}
}
]
}
},
"roleName": "lambda_role_vault_lab",
"trustedEntities": [
"lambda.amazonaws.com"
]
}
I don't include the python code has the lambda crashs before starting executing it.
For info, I tried a few changes in the policy without any effects.
Has someone an idea where I did go wrong?
Thanks, :)
David

Following required service principals [ec2.amazonaws.com] were not found in the trust relationships / AWSServiceRoleForAmazonEKSNodegroup

I have EKS cluster
{
"cluster": {
"name": "cluster",
"arn": "arn:aws:eks:us-west-2:xxx:cluster/cluster",
"createdAt": "2022-10-04T18:24:38.786000-07:00",
"version": "1.23",
"endpoint": "https://xxx.gr7.us-west-2.eks.amazonaws.com",
"roleArn": "arn:aws:iam::xxx:role/eksrole",
"resourcesVpcConfig": {
"subnetIds": [
"subnet-08f0fc71",
"subnet-53032a18"
],
"securityGroupIds": [],
"clusterSecurityGroupId": "sg-06fa1aa90566a372e",
"vpcId": "vpc-0e09ba76",
"endpointPublicAccess": true,
"endpointPrivateAccess": false,
"publicAccessCidrs": [
"0.0.0.0/0"
]
},
"kubernetesNetworkConfig": {
"serviceIpv4Cidr": "10.100.0.0/16",
"ipFamily": "ipv4"
},
"logging": {
"clusterLogging": [
{
"types": [
"api",
"audit",
"authenticator",
"controllerManager",
"scheduler"
],
"enabled": false
}
]
},
"identity": {
"oidc": {
"issuer": "https://oidc.eks.us-west-2.amazonaws.com/id/18BD011EE7130A4BA313C555E62FB9FA"
}
},
"status": "ACTIVE",
xxx
},
"platformVersion": "eks.2",
"tags": {}
}
}
It has 1 node group which I added in UI (my account shown in UI arn:aws:iam::xxx9399:user/user)
aws eks list-nodegroups --cluster-name cluster
{
"nodegroups": [
"node-group"
]
}
I have no problems adding more node groups to the same cluster in UI, but when I use AWS CLI I get this error:
aws eks create-nodegroup --cluster-name cluster --nodegroup-name NG --subnets subnet-08f0fc71 subnet-53032a18 --node-role arn:aws:iam::xxx9399:role/aws-service-role/eks-nodegroup.amazonaws.com/AWSServiceRoleForAmazonEKSNodegroup
An error occurred (InvalidParameterException) when calling the CreateNodegroup operation: Following required service principals [ec2.amazonaws.com] were not found in the trust relationships of nodeRole arn:aws:iam::xxx9399:role/aws-service-role/eks-nodegroup.amazonaws.com/AWSServiceRoleForAmazonEKSNodegroup
I checked AWSServiceRoleForAmazonEKSNodegroup
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "SharedSecurityGroupRelatedPermissions",
"Effect": "Allow",
"Action": [
"ec2:RevokeSecurityGroupIngress",
"ec2:AuthorizeSecurityGroupEgress",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:DescribeInstances",
"ec2:RevokeSecurityGroupEgress",
"ec2:DeleteSecurityGroup"
],
"Resource": "*",
"Condition": {
"StringLike": {
"ec2:ResourceTag/eks": "*"
}
}
},
{
"Sid": "EKSCreatedSecurityGroupRelatedPermissions",
"Effect": "Allow",
"Action": [
"ec2:RevokeSecurityGroupIngress",
"ec2:AuthorizeSecurityGroupEgress",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:DescribeInstances",
"ec2:RevokeSecurityGroupEgress",
"ec2:DeleteSecurityGroup"
],
"Resource": "*",
"Condition": {
"StringLike": {
"ec2:ResourceTag/eks:nodegroup-name": "*"
}
}
},
{
"Sid": "LaunchTemplateRelatedPermissions",
"Effect": "Allow",
"Action": [
"ec2:DeleteLaunchTemplate",
"ec2:CreateLaunchTemplateVersion"
],
"Resource": "*",
"Condition": {
"StringLike": {
"ec2:ResourceTag/eks:nodegroup-name": "*"
}
}
},
{
"Sid": "AutoscalingRelatedPermissions",
"Effect": "Allow",
"Action": [
"autoscaling:UpdateAutoScalingGroup",
"autoscaling:DeleteAutoScalingGroup",
"autoscaling:TerminateInstanceInAutoScalingGroup",
"autoscaling:CompleteLifecycleAction",
"autoscaling:PutLifecycleHook",
"autoscaling:PutNotificationConfiguration",
"autoscaling:EnableMetricsCollection"
],
"Resource": "arn:aws:autoscaling:*:*:*:autoScalingGroupName/eks-*"
},
{
"Sid": "AllowAutoscalingToCreateSLR",
"Effect": "Allow",
"Condition": {
"StringEquals": {
"iam:AWSServiceName": "autoscaling.amazonaws.com"
}
},
"Action": "iam:CreateServiceLinkedRole",
"Resource": "*"
},
{
"Sid": "AllowASGCreationByEKS",
"Effect": "Allow",
"Action": [
"autoscaling:CreateOrUpdateTags",
"autoscaling:CreateAutoScalingGroup"
],
"Resource": "*",
"Condition": {
"ForAnyValue:StringEquals": {
"aws:TagKeys": [
"eks",
"eks:cluster-name",
"eks:nodegroup-name"
]
}
}
},
{
"Sid": "AllowPassRoleToAutoscaling",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "*",
"Condition": {
"StringEquals": {
"iam:PassedToService": "autoscaling.amazonaws.com"
}
}
},
{
"Sid": "AllowPassRoleToEC2",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "*",
"Condition": {
"StringEqualsIfExists": {
"iam:PassedToService": [
"ec2.amazonaws.com",
"ec2.amazonaws.com.cn"
]
}
}
},
{
"Sid": "PermissionsToManageResourcesForNodegroups",
"Effect": "Allow",
"Action": [
"iam:GetRole",
"ec2:CreateLaunchTemplate",
"ec2:DescribeInstances",
"iam:GetInstanceProfile",
"ec2:DescribeLaunchTemplates",
"autoscaling:DescribeAutoScalingGroups",
"ec2:CreateSecurityGroup",
"ec2:DescribeLaunchTemplateVersions",
"ec2:RunInstances",
"ec2:DescribeSecurityGroups",
"ec2:GetConsoleOutput",
"ec2:DescribeRouteTables",
"ec2:DescribeSubnets"
],
"Resource": "*"
},
{
"Sid": "PermissionsToCreateAndManageInstanceProfiles",
"Effect": "Allow",
"Action": [
"iam:CreateInstanceProfile",
"iam:DeleteInstanceProfile",
"iam:RemoveRoleFromInstanceProfile",
"iam:AddRoleToInstanceProfile"
],
"Resource": "arn:aws:iam::*:instance-profile/eks-*"
},
{
"Sid": "PermissionsToManageEKSAndKubernetesTags",
"Effect": "Allow",
"Action": [
"ec2:CreateTags",
"ec2:DeleteTags"
],
"Resource": "*",
"Condition": {
"ForAnyValue:StringLike": {
"aws:TagKeys": [
"eks",
"eks:cluster-name",
"eks:nodegroup-name",
"kubernetes.io/cluster/*"
]
}
}
}
]
}
It shows the following Trusted entities
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "eks-nodegroup.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
I cannot add anything to Trust Relationships (entities are not editable)
I also have AWSServiceRoleForAmazonEKSNodegroup Policy with attached AWSServiceRoleForAmazonEKSNodegroup role to it
aws iam get-policy --policy-arn arn:aws:iam::aws:policy/aws-service-role/AWSServiceRoleForAmazonEKSNodegroup
{
"Policy": {
"PolicyName": "AWSServiceRoleForAmazonEKSNodegroup",
"PolicyId": "ANPAZKAPJZG4KH2AAMJJG",
"Arn": "arn:aws:iam::aws:policy/aws-service-role/AWSServiceRoleForAmazonEKSNodegroup",
"Path": "/aws-service-role/",
"DefaultVersionId": "v6",
"AttachmentCount": 1,
"PermissionsBoundaryUsageCount": 0,
"IsAttachable": true,
"Description": "Permissions required for managing nodegroups in the customer's account. These policies related to management of the following resources: AutoscalingGroups, SecurityGroups, LaunchTemplates and InstanceProfiles.",
"CreateDate": "2019-11-07T01:34:26+00:00",
"UpdateDate": "2022-01-14T00:33:26+00:00",
"Tags": []
}
}
But I cannot attach it to my user
I would like to know what actions I need to perform to
be able to create a managed node group
add required service principals [ec2.amazonaws.com] in the trust relationships of nodeRole arn:aws:iam::xxx9399:role/aws-service-role/eks-nodegroup.amazonaws.com/AWSServiceRoleForAmazonEKSNodegroup
Thanks
You need to change eks-nodegroup.amazonaws.com to ec2.amazonaws.com in the permissions. At least that was my problem.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "eks-nodegroup.amazonaws.com" // change to ec2.amazonaws.com
},
"Action": "sts:AssumeRole"
}
]
}

When creating a Cross Account Log Destination deploy fails with "Could not deliver test message to specified destination"

I am trying to create the cross account log data sharing as per the mentioned link using cloudformation
When I delete the destination Part, everything is being created as normal.
https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CreateFirehoseStream.html
"Resources": {
"KdsDataStream01BCE77D": {
"Type": "AWS::Kinesis::Stream",
"Properties": {
"Name": "RecipientStream",
"ShardCount": 1,
"RetentionPeriodHours": 24
}
},
"CWLtoKinesisRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Principal": {
"Service": "logs.amazonaws.com"
},
"Condition": {
"StringLike": {
"aws:SourceArn": [
{
"Ref" : "SourceAccountId",
"Ref" : "RecipientAccountId"
}
]
}
},
"Action": "sts:AssumeRole"
}
},
"Policies": [
{
"PolicyName": "Permissions-Policy-For-CWL",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "kinesis:PutRecords",
"Resource": [
{
"Fn::GetAtt": [
"KdsDataStream01BCE77D",
"Arn"
]
}
]
}
]
}
}
]
}
},
"ServiceRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"firehose.amazonaws.com",
"logs.amazonaws.com",
"lambda.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
},
"Policies": [
{
"PolicyName": "kinesis-access",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"kinesis:PutRecords",
"kinesis:GetRecords",
"kinesis:SubscribeToShard",
"kinesis:ListShards",
"kinesis:ListStreams",
"kinesis:DescribeStreamSummary",
"kinesis:DescribeLimits",
"kinesis:DescribeStream",
"kinesis:ListStreamConsumers",
"kinesis:DescribeStreamConsumer",
"kinesis:GetShardIterator",
"kinesis:ListTagsForStream",
"Lambda:InvokeFunction"
],
"Resource": "*"
}
]
}
},
{
"PolicyName": "firehose-access",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"firehose:DescribeDeliveryStream",
"firehose:ListDeliveryStreams",
"firehose:ListTagsForDeliveryStream"
],
"Resource": "*"
}
]
}
},
{
"PolicyName": "s3-access",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:*Object",
"s3:ListBucket",
"s3:ListBucket",
"s3:GetBucketLocation",
"s3:ListAllMyBuckets"
],
"Resource": "*"
}
]
}
},
{
"PolicyName": "cloudwatch-access",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"cloudwatch:*",
"logs:Describe*",
"logs:Delete*",
"logs:Get*",
"logs:Put*"
],
"Effect": "Allow",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "iam:CreateServiceLinkedRole",
"Resource": "arn:aws:iam::*:role/aws-service-role/events.amazonaws.com/AWSServiceRoleForCloudWatchEvents*",
"Condition": {
"StringLike": {
"iam:AWSServiceName": "events.amazonaws.com"
}
}
}
]
}
},
{
"PolicyDocument": {
"Statement": [
{
"Action": [
"kinesis:DescribeStream",
"kinesis:DescribeStreamSummary",
"kinesis:GetShardIterator",
"kinesis:GetRecords",
"kinesis:ListShards",
"kinesis:SubscribeToShard"
],
"Effect": "Allow",
"Resource": [
{
"Fn::GetAtt": [
"KdsDataStream01BCE77D",
"Arn"
]
}
]
}
],
"Version": "2012-10-17"
},
"PolicyName": "ReadSource"
}
],
"RoleName": "ServiceRole"
}
},
"DestinationWithName" : {
"Type" : "AWS::Logs::Destination",
"Properties" : {
"DestinationName": "TestDestination",
"RoleArn": {
"Fn::GetAtt": [
"CWLtoKinesisRole",
"Arn"
]
},
"TargetArn": {
"Fn::GetAtt": [
"KdsDataStream01BCE77D",
"Arn"
]
},
"DestinationPolicy": "{\"Version\" : \"2012-10-17\",\"Statement\" : [{\"Effect\" : \"Allow\", \"Principal\" : {\"AWS\" : \"*\"},\"Action\" : \"logs:PutSubscriptionFilter\", \"Resource\" : \"arn:aws:logs:us-east-1:763475694844:destination:TestDestination\"}]}"
}
},
SourceAccountID and RecipientAccountID are passed during runtime.
Problem that I am facing is , it shows the error as
When creating a Cross Account Log Destination deploy fails with "Could not deliver test message to specified destination". (Invalid Parameter Exception)
How can I resolve this issue

Cross-Account ECS Deployment

I am trying to deploy an image from ECR of one account (AccountA) to ECS Cluster of another (AccountB) using CodePipeline. I am getting a permissions related error in the deploy phase.
Here is my pipeline role in AccountA:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:GetObject",
"s3:GetObjectVersion",
"s3:GetBucketVersioning"
],
"Resource": "*",
"Effect": "Allow"
},
{
"Action": [
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::<bucketname>/*"
],
"Effect": "Allow"
},
{
"Action": [
"ecr:GetAuthorizationToken",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
"ecr:GetRepositoryPolicy",
"ecr:DescribeRepositories",
"ecr:ListImages",
"ecr:DescribeImages",
"ecr:BatchGetImage",
"ecr:InitiateLayerUpload",
"ecr:UploadLayerPart",
"ecr:CompleteLayerUpload",
"ecr:PutImage"
],
"Resource": [
"*"
],
"Effect": "Allow"
},
{
"Action": [
"codebuild:BatchGetBuilds",
"codebuild:InvalidateProjectCache",
"codebuild:StartBuild",
"codebuild:StopBuild",
"codebuild:UpdateProject",
"codebuild:UpdateWebhook"
],
"Resource": "*",
"Effect": "Allow"
},
{
"Action": [
"sts:AssumeRole"
],
"Resource": "arn:aws:iam::<AccountB>:role/taskexecutionrole",
"Effect": "Allow"
}
]
}
The arn:aws:iam::<AccountB>:role/taskexecutionrole role exists in AccountB and trusts AccountA. Here is the role in AccountB:
{
"Effect": "Allow",
"Action": "ecs:*",
"Resource": [
"*"
]
}
The pipeline has a ECR source, build stage generates an imagedefinitions.json file. And finally the deployment stage does ECS deploy.
The error I am getting is:
Invalid action configuration
Identifier is for AccountB. Your accountId is AccountA
This answer helps only for manual CLI deployment and I have tried the solution from this answer.
Any pointers what I'm missing?
Lets assume:
Account_A => CodePipeline & Source
Account_B => ECS
Here is what is required:
Account_A:
* AWSCodePipelineServiceRole
* Artifact_Store_S3_Bucket
* KMS_Key_for_Pipeline_Artifact (Customer Managed Key)
* Bucket Policy on Artifact_Store_S3_Bucket to Allow Account_B access
* Key Policy on KMS_Key_for_Pipeline_Artifact to Allow Access to Cross_Account_Role (from Account_B)
Account_B
* Cross_Account_Role (Trust relationship with Account_A and Full_ECS permissions)
* ECS with a running that is to be replaced with deployment
imagedefinitions.json (must be part of your source code)
[
{
"name": "container_name",
"imageUri": "nginx:latest"
}
]
Bucket_Policy on Artifact_Store_S3_Bucket
{
"Version": "2012-10-17",
"Id": "SSEAndSSLPolicy",
"Statement": [
{
"Sid": "DenyUnEncryptedObjectUploads",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::Artifact_Store_S3_Bucket/*",
"Condition": {
"StringNotEquals": {
"s3:x-amz-server-side-encryption": "aws:kms"
}
}
},
{
"Sid": "DenyInsecureConnections",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": "arn:aws:s3:::Artifact_Store_S3_Bucket/*",
"Condition": {
"Bool": {
"aws:SecureTransport": "false"
}
}
},
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::Account_B:root"
},
"Action": [
"s3:Get*",
"s3:Put*"
],
"Resource": "arn:aws:s3:::Artifact_Store_S3_Bucket/*"
},
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::Account_B:root"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::Artifact_Store_S3_Bucket"
}
]
}
pipeline.json:
{
"pipeline": {
"name": "test",
"roleArn": "arn:aws:iam::Account_A:role/service-role/AWSCodePipelineServiceRole",
"artifactStore": {
"type": "S3",
"location": "Artifact_Store_S3_Bucket",
"encryptionKey": {
"id": "arn:aws:kms:us-east-1:Account_A:key/KMS_Key_for_Pipeline_Artifact",
"type": "KMS"
}
},
"stages": [
{
"name": "Source",
"actions": [
{
"name": "Source",
"actionTypeId": {
"category": "Source",
"owner": "AWS",
"provider": "CodeCommit",
"version": "1"
},
"runOrder": 1,
"configuration": {
"BranchName": "master",
"PollForSourceChanges": "false",
"RepositoryName": "code"
},
"outputArtifacts": [
{
"name": "SourceArtifact"
}
],
"inputArtifacts": [],
"region": "us-east-1"
}
]
},
{
"name": "Deploy",
"actions": [
{
"name": "Deploy",
"actionTypeId": {
"category": "Deploy",
"owner": "AWS",
"provider": "ECS",
"version": "1"
},
"runOrder": 1,
"roleArn": "arn:aws:iam::Account_B:role/CrossAccount_Role",
"configuration": {
"ClusterName": "<Cluster>",
"ServiceName": "<Service>"
},
"outputArtifacts": [],
"inputArtifacts": [
{
"name": "SourceArtifact"
}
],
"region": "us-east-1"
}
]
}
],
"version": 1
}
}
To Update the Pipeline:
$ aws codepipeline update-pipeline --region us-east-1 --cli-input-json file://pipeline.json

Incorrect S3 bucket policy is detected for bucket in CloudFormation

I have issues implementing CloudTrail via Cloudformation, with a Incorrect S3 bucket policy is detected for bucket error being thrown when I try to launch the model.
Here is the configuration from the BucketPolicy:
"LogBucketPolicy": {
"Type": "AWS::S3::BucketPolicy",
"Properties": {
"Bucket": {
"Ref": "LogBucket"
},
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSCloudTrailAclCheck",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:s3:::",
{
"Ref": "LogBucket"
}
]
]
}
},
{
"Sid": "AWSCloudTrailWrite",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:s3:::",
{
"Ref": "LogBucket"
},
"/AWSLogs/139339407673/*"
]
]
},
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
}
]
}
}
}
I have copied the template from AWS examples, but let me know if I did a mistake in the implementation.
Edit: The error is not thrown by the bucket policy, but by CloudTrail. Here is the configuration of the bucket:
"Trail": {
"Type": "AWS::CloudTrail::Trail",
"Properties": {
"SnsTopicName": {
"Fn::GetAtt": [
"Topic",
"TopicName"
]
},
"IsLogging": true,
"S3BucketName": {
"Ref": "LogBucket"
}
},
"DependsOn": [
"LogBucket"
]
}
As Krishna has mentioned, the error came from the fact that I didn't put the dependence of the BucketPolicy. When this was done, the stack was deployed with no issues.
I modified your code and it seems to be working for me. Can you try this?
{
"Parameters": {
"LogBucket": {
"Description": "Name Bucket.",
"Type": "String"
}
},
"Resources": {
"LogBucketPolicy": {
"Type": "AWS::S3::BucketPolicy",
"Properties": {
"Bucket": {
"Ref": "LogBucket"
},
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSCloudTrailAclCheck",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:s3:::",
{
"Ref": "LogBucket"
}
]
]
}
},
{
"Sid": "AWSCloudTrailWrite",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:s3:::",
{
"Ref": "LogBucket"
},
"/AWSLogs/139339407673/*"
]
]
},
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
}
]
}
}
}
}
}
Beside the dependency issue mentioned in the accepted answer, the error can also comes from different cases of wrong configuration of the S3 policy.
For example, if we look at the following policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSCloudTrailAclCheck20131101",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": "arn:aws:s3:::myBucketName"
},
{
"Sid": "AWSCloudTrailWrite20131101",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::myBucketName/[optional] myLogFilePrefix/AWSLogs/<account-id>/*"
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
}
]
}
Looking at the Resource block of the 2nd statement:
"Resource": "arn:aws:s3:::myBucketName/[optional] myLogFilePrefix/AWSLogs/<account-id>/*"
Passing wrong values to the Resource block like bad prefix (my case) or forgetting the "*" postfix (like mentioned here in last scenario) can lead to the error.
(*) Example taken from here.