I am attempting to deploy a CloudFormation template that pulls in some parameters from SSM using the method described in this blog-post: https://aws.amazon.com/blogs/mt/integrating-aws-cloudformation-with-aws-systems-manager-parameter-store/
The relevant excerpt from the Parameters section of the CF template is:
"ZoneName" : {
"Type" : "AWS::SSM::Parameter::Value<String>",
"Description" : "DNS Hostname Zone",
"Default" : "/Deimos/ZoneName"
},
"ZoneId" : {
"Type" : "AWS::SSM::Parameter::Value<String>",
"Description" : "DNS Hostname Zone",
"Default" : "/Deimos/ZoneId"
},
However, I'm getting the following error when I attempt to deploy it (via CodePipeline):
Action execution failed
AccessDenied. User doesn't have permission to call ssm:GetParameters (Service: AmazonCloudFormation; Status Code: 400; Error Code: ValidationError; Request ID: d6756fbe-fd41-4ac5-93bd-56e5b397445e)
I've got a Role and Policy setup for CloudFormation that includes the following section to grant access to some parameter namespaces within SSM:
{
"Sid": "XonoticCFFetchParameters",
"Effect": "Allow",
"Action": [
"ssm:GetParameters",
"ssm:GetParameter"
],
"Resource": [
"arn:aws:ssm:*:<aws account #>:parameter/Deimos/*",
"arn:aws:ssm:*:<aws account #>:parameter/Installers/*",
"arn:aws:ssm:*:<aws account #>:parameter/Xonotic/*"
]
},
These seem to have been applied just fine, based on the use of
aws iam simulate-principal-policy --policy-source-arn "arn:aws:iam::<aws account #>:role/Xonotic-CloudFormationDeploy" --action-names "ssm:getParameters" --resource-arns "arn:aws:ssm:*:<aws account #>:parameter/Deimos/ZoneName"
{
"EvaluationResults": [
{
"EvalActionName": "ssm:getParameters",
"EvalResourceName": "arn:aws:ssm:*:<aws account #>:parameter/Deimos/ZoneName",
"EvalDecision": "allowed",
"MatchedStatements": [
{
"SourcePolicyId": "Xonotic-Deployment",
"StartPosition": {
"Line": 3,
"Column": 19
},
"EndPosition": {
"Line": 16,
"Column": 10
}
}
],
"MissingContextValues": []
}
]
}
So, the Role I'm using should have the access needed to fetch the parameter in question, but it's not working and I'm out of things to check.
Ok - so in this case it turns out there was a JSON parameters file that was part of the build pipeline that was overriding one of my parameters with an invalid value (it was putting the actual zone name in ZoneName).
Fixed that and parameters are now being passed to my build process just fine.
Related
I use CDK to deploy a codepipeline. It works fine until I try to add notification for codepipeline success/fail events. It gives CREATE_FAILED error with message Resource handler returned message: "Invalid request provided: AWS::CodeStarNotifications::NotificationRule" (RequestToken: bb566fd0-1ac9-5d61-03fe-f9c27b4196fa, HandlerErrorCode: InvalidRequest). What could be the reason? Thanks.
import * as codepipeline from "#aws-cdk/aws-codepipeline";
import * as codepipeline_actions from "#aws-cdk/aws-codepipeline-actions";
import * as codestar_noti from "#aws-cdk/aws-codestarnotifications";
import * as sns from "#aws-cdk/aws-sns";
const pipeline = new codepipeline.Pipeline(...);
const topicArn = props.sns_arn_for_developer;
const targetTopic = sns.Topic.fromTopicArn(
this,
"sns-notification-topic",
topicArn
);
new codestar_noti.NotificationRule(this, "Notification", {
detailType: codestar_noti.DetailType.BASIC,
events: [
"codepipeline-pipeline-pipeline-execution-started",
"codepipeline-pipeline-pipeline-execution-failed",
"codepipeline-pipeline-pipeline-execution-succeeded",
"codepipeline-pipeline-pipeline-execution-canceled",
],
source: pipeline,
targets: [targetTopic],
});
Here is the snippet of generated cloudformation tempalte.
"Notification2267453E": {
"Type": "AWS::CodeStarNotifications::NotificationRule",
"Properties": {
"DetailType": "BASIC",
"EventTypeIds": [
"codepipeline-pipeline-pipeline-execution-started",
"codepipeline-pipeline-pipeline-execution-failed",
"codepipeline-pipeline-pipeline-execution-succeeded",
"codepipeline-pipeline-pipeline-execution-canceled"
],
"Name": "sagemakerbringyourownNotification36194CEC",
"Resource": {
"Fn::Join": [
"",
[
"arn:",
{
"Ref": "AWS::Partition"
},
":codepipeline:ap-southeast-1:305326993135:",
{
"Ref": "sagemakerbringyourownpipeline0A8C43B1"
}
]
]
},
"Targets": [
{
"TargetAddress": "arn:aws:sns:ap-southeast-1:305326993135:whitespace_alerts",
"TargetType": "SNS"
}
]
},
"Metadata": {
"aws:cdk:path": "sagemaker-bring-your-own/Notification/Resource"
}
},
FWIW, I got the exact same error "Invalid request provided: AWS::CodeStarNotifications::NotificationRule" from a CDK app where the Topic was created (not imported). It turned out to be a transient issue, because it succeeded the second time without any changes. I suspect it was due to a very large ECR image which was build the first time as part of the deploy and which took quite some time. My speculation is that the Topic timed out and got into some kind of weird state waiting for the NotificationRule to be created.
This is because imported resources cannot be modified. As you pointed out in the comments, setting up the notification involves modifying the Topic resource, specifically its access policy.
Reference: https://docs.aws.amazon.com/cdk/v2/guide/resources.html#resources_importing
I was able to solve this by doing the following in that order:
First removing the below statement from the resource policy of the SNS topic.
Then deploying the stack(which interestingly doesn't add anything to the resource policy)
Once the stack deployment finishes, update the resource policy manually to add the below statement.
{
"Sid": "AWSCodeStarNotifications_publish",
"Effect": "Allow",
"Principal": {
"Service": "codestar-notifications.amazonaws.com"
},
"Action": "SNS:Publish",
"Resource": "arn:aws:sns:ap-south-1:xxxxxxxxx:test"
}
Normally if I want to create a private AWS::ApiGateway::RestApi with policy only allowing VPC traffic to invoke any resources on the API I'd do something like so:
"ApiGatewayRestApi": {
"Type": "AWS::ApiGateway::RestApi",
"Properties": {
"Name": "api-foo-bar",
"EndpointConfiguration": {
"Types": [
"PRIVATE"
]
},
"Policy": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": [
"execute-api:Invoke"
],
"Resource": "execute-api:/*",
"Condition": {
"StringEquals": {
"aws:SourceVpc": "vpc-000000000000"
}
}
}
]
}
}
}
I have been asked if we can create a policy and then reuse it for different Api Gateways we may create? something amongst the lines:
"ApiGatewayRestApi": {
"Type": "AWS::ApiGateway::RestApi",
"Properties": {
"Name": "api-foo-bar",
"EndpointConfiguration": {
"Types": [
"PRIVATE"
]
},
"Policy": "arn:aws:*whatever*"
}
},
And I have no idea! Nor can I find any documentation or examples showing that. Has anyone done this? Is it at all doable? Thanks :)
No, it's not doable currently. The policy you are attaching to the Api Gateway is Resource-based policy.
From aws documentation,
With resource-based policies, you can specify who has access to the
resource and what actions they can perform on it.
Aws documentation shows the type that each property in cloudformation can take. The following is the properties and the types allowed in "AWS::ApiGateway::RestApi"
{
"Type" : "AWS::ApiGateway::RestApi",
"Properties" : {
"ApiKeySourceType" : String,
"BinaryMediaTypes" : [ String, ... ],
"Body" : Json,
"BodyS3Location" : S3Location,
"CloneFrom" : String,
"Description" : String,
"DisableExecuteApiEndpoint" : Boolean,
"EndpointConfiguration" : EndpointConfiguration,
"FailOnWarnings" : Boolean,
"MinimumCompressionSize" : Integer,
"Mode" : String,
"Name" : String,
"Parameters" : {Key : Value, ...},
"Policy" : Json,
"Tags" : [ Tag, ... ]
}
}
Notice that Policy property takes the type of JSON. Furthermore, the documentation writes the following for the Policy property:
A policy document that contains the permissions for the RestApi
resource.
and gives us a hint that Policy property does not take the following form: "Policy": "arn:aws:*whatever*" and only accepts a policy document in the form of JSON as Api Gateway's resource-based policy.
ref:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-apigateway-restapi.html#cfn-apigateway-restapi-policy
https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_identity-vs-resource.html
you can find the docs here.
Policy
A policy document that contains the permissions for the RestApi resource. To set the ARN for the policy, use the !Join intrinsic function with "" as delimiter and values of "execute-api:/" and "*".
I noticed that LakeFormation SDK calls (both boto3 via AWS CLI, and Go via Terraform) are returning ALL in the list of permissions assigned to a resource.
for example:
"PrincipalResourcePermissions": [
{
"Principal": {
"DataLakePrincipalIdentifier": "arn:aws:iam::ACCOUNT:role/FooRole"
},
"Resource": {
"Table": {
"CatalogId": "ACCOUNT",
"DatabaseName": "lf_test",
"Name": "foo"
}
},
"Permissions": [
"ALL",
"DESCRIBE"
],
"PermissionsWithGrantOption": []
}
Yet, I cannot delete this "ALL" permission. Attempting to revoke with either AWS CLI or Terraform results in error:
An error occurred (InvalidInputException) when calling the RevokePermissions operation: No permissions revoked. Grantee does not have:[ALL]
What's going on here, and how do I fix it other than special-casing to ignore "ALL"?
I am using serverless to manage IaC which uses cloudformation internally. There are a number of lambdas and roles defined in serverless.yml. It works very well until I tried to sls remove all resources and sls deploy again. After doing that, I get an error when run lambdas: The role defined for the function cannot be assumed by Lambda. (Service: AWSLambda; Status Code: 403; Error Code: AccessDeniedException; Request ID: 0879c203-bec7-480b-81c6-4c7a61e2cb15.
The error says lambda doesn't have permission however, it works if I change the lambda role to something else and change it back. It seems that lambda's role still references to the deprecated one.
I wonder what the proper way to do remove followed by deploy.
The role is:
{
"Role": {
"Path": "/",
"RoleName": "getSiteHandlerRole",
"RoleId": "xxxxx",
"Arn": "arn:aws:iam::115136697128:role/getSiteHandlerRole",
"CreateDate": "2020-07-27T03:37:18Z",
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
},
"Description": "",
"MaxSessionDuration": 3600,
"Tags": [
{
"Key": "STAGE",
"Value": "user"
}
]
}
}
I've two AWS Cloudformation stacks, one for IAM roles and the second to create an AWS service and import the respective roles into it using Cloudformation.
When 10+ services are deployed the following error appears randomly on 1 or 2 of the services -
AWS::ECS::Service service Unable to assume role and validate the
listeners configured on your load balancer. Please verify that the ECS
service role being passed has the proper permissions.
If all the services are torn down and the services redployed to the ECS cluster, the error appears but for different services.
The AWS fix for this can be seen here
If the 1 or 2 broken services are torn down and redeployed the services deploy without issue. So the problem appears to only occur when many services are deployed at the same time - this indicates it may be an IAM propagation timing issue within Cloudformation.
I've tried adding depends on in the service definition -
"service" : {
"Type" : "AWS::ECS::Service",
"DependsOn" : [
"taskdefinition",
"ECSServiceRole"
],
"Properties" : {
"Cluster" : { "Ref": "ECSCluster"},
"Role" : {"Ref" : "ECSServiceRole"},
etc...
}
}
But this doesn't work.
As you can note, I've also removed the IAM import value for the ECSServiceRole and replaced it with an inline resource policy seen here -
"ECSServiceRole" : {
"Type" : "AWS::IAM::Role",
"Properties" : {
"AssumeRolePolicyDocument" : {
"Statement" : [
{
"Sid": "",
"Effect" : "Allow",
"Principal" : {
"Service" : [
"ecs.amazonaws.com"
]
},
"Action" : [
"sts:AssumeRole"
]
}
]
},
"Path" : "/",
"Policies" : [
{
"PolicyName" : "ecs-service",
"PolicyDocument" : {
"Statement" : [
{
"Effect" : "Allow",
"Action" : [
"ec2:Describe*",
"ec2:AuthorizeSecurityGroupIngress",
"elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
"elasticloadbalancing:DeregisterTargets",
"elasticloadbalancing:Describe*",
"elasticloadbalancing:RegisterInstancesWithLoadBalancer",
"elasticloadbalancing:RegisterTargets",
"sns:*"
],
"Resource" : "*"
}
]
}
}
]
}
}
But again - the inline policy doesn't fix the issue either.
Any ideas or pointers would be much appreciated!
In reply to answer 1.
Thank you - I wasn't aware of this improvment.
Is this the correct way to associate the service linked role for ECS?
"ECSServiceRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": [
"ecs.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Path": "/",
"Policies": [
{
"PolicyName": "CreateServiceLinkedRoleForECS",
"PolicyDocument": {
"Statement": [
{
"Effect": "Allow",
"Action": [
"iam:CreateServiceLinkedRole",
"iam:PutRolePolicy",
"iam:UpdateRoleDescription",
"iam:DeleteServiceLinkedRole",
"iam:GetServiceLinkedRoleDeletionStatus"
],
"Resource": "arn:aws:iam::*:role/aws-service-role/ecs.amazonaws.com/AWSServiceRoleForECS*",
"Condition": {
"StringLike": {
"iam:AWSServiceName": "ecs.amazonaws.com"
}
}
}
]
}
}
]
}
}
Final Answer
After months of intermittent on-going issues with AWS regarding this matter AWS came back to say they were throttling us in the background, on the ELB. This is why the random and varied issues were appearing when deploying 3+ docker services via Cloudformation at the same time. The solution was nothing to do with IAM permissions, rather it was to increase the rate limit on the ELB via the "AWS Service Team".
So the fix was to continue using the two stack approach in Cloudformation, one with the IAM roles, which in turn were imported into the service layer stack. The fix was to add a depends on in the service definition for all of the other stack resources in the service layer script. By doing this it allows sufficient time for the IAM roles to be imported and executed by the service, thus this was a Cloudformation resource creation timing issue.
"service" : {
"Type" : "AWS::ECS::Service",
"DependsOn" : [
"TaskDefinition",
"EcsElasticLoadBalancer",
"DnsRecord"
],
"Properties" : {
etc...
}
}
UPDATE:
As of July 19th 2018, it is now possible to create a IAM Service-Linked Roles using CloudFormation https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-servicelinkedrole.html.
EcsServiceLinkedRole:
Type: "AWS::IAM::ServiceLinkedRole"
Properties:
AWSServiceName: "ecs.amazonaws.com"
Description: "Role to enable Amazon ECS to manage your cluster."
OLD ANSWER:
Creating your own ECSServiceRole is no longer required. By not specifying a role for your service, AWS will default on using the ECS Service-Linked role. If your AWS account is recent enough, or you have already created a cluster via the console you don't have to do anything for this to work. If not, run the following command to create the role: aws iam create-service-linked-role --aws-service-name ecs.amazonaws.com.