I have external-secrets operator v0.5.1 installed and working with a SecretStore for retrieve AWS parameter store. Also tried updating to V0.5.8
This is working fine with IRSA but if I try to create a external-secret for AWS secrets, with a new SecretStore, the SecretStore' status is Valid but the ExternalSecret that references this SecretStore got the following error: SecretSyncedError
AccessDeniedException: User: arn:aws:sts::12345678:assumed-role/eks-backend-role-pre/external-secrets-provider-aws is not authorized to perform: secretsmanager:GetSecretValue on resource: /backend/pre/PRE_PRIVPGPKEY because no identity-based policy allows the secretsmanager:GetSecretValue action status code: 400,
Please, note the STS is trying to use eks-backend-role-pre/external-secrets-provider-aws which it doesn't exist. The role which exist is eks-backend-role-pre I'm not sure who is adding the suffix external-secrets-provider-aws which invalidate the role name.
Both SecretStore, the one dedicated to AWS Parameter Store and the other that is dedicated to gather from AWS Secrets has the same service account associated.
Why is working one External Secret and the other using the same service account don't?
There was a typo in the policy.
As documentation shows, this is the correct way to declare a principal:
arn:${Partition}:secretsmanager:${Region}:${Account}:secret:${SecretId}
I have incorrect declared the principal:
"arn:aws:secretsmanager:eu-west-1:1234567890:secret/backend/pre/*"
Correct:
"arn:aws:secretsmanager:eu-west-1:1234567890:secret:/backend/pre/*"
In contrast to SSM Parameter store that you declare your principal "arn:aws:ssm:eu-west-1:1234567890:parameter/backend/pre/*"
, with SecretsManager you need to add a colon after the service :secret:
Related
I am trying to create aws eks cluster with terraform. I want to use an existing iam role. I won't be creating a new role. I tried this.
module 'eks' {
...
iam_role_arn = "arn:aws:iam::11111111:role/cluster-role"
...
}
But I get 403 permission denied. How can I this? Thanks.
Edit: added error.
failed creating IAM Role (green-eks-node-group): AccessDenied: User: arn:aws:iam::*******:user/****** is not authorized to perform: iam:TagRole on resource: arn:aws:iam::*******:role/green-eks-node
First for good measure: Please do not post pictures here of either code or error messages. Just copy paste the stuff and format it.
That said: I think the error tells you exactly what is wrong (or at least what the first error is, because in my experience it takes a few iterations of this to get your IAM permissions straight).
The role that you use you execute terraform does not have the permission:
iam:TagRole
on the resource:
arn:aws:iam:xxxxxxxxxxx:role/eks-cluster
So you will need to add it to it's policy.
First time deploying an app to AWS and it's been an adventure. My current error:
Dec 21 03:49:33 ip-172-31-31-185 web: botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the GetParameter operation: User: arn:aws:sts::[my account number]:assumed-role/aws-elasticbeanstalk-ec2-role/[the instance number] is not authorized to perform: ssm:GetParameter on resource: arn:aws:ssm:us-east-2:[my account number]:parameter/Dev/WebServer/[thing I want to get] because no identity-based policy allows the ssm:GetParameter action
The docs are very good at telling me what I need to do-- it appears I need to add an IAM policy to the instance using AWS systems manager. I even have the policy-- but less good in telling me how to do it. Where can you add instance policies in AWS Console?
Based on the error message - it comes from Beanstalk, the documentation is https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/concepts-roles-instance.html
So likely you already have an instance profile based on an existing role. So that is the role you need to change. The policy is managed in IAM (not Systems Manager), just add ssm:GetParameter rule there either with a star (which may be good enough to debug in the dev environment) and if it works then limit it to your arn, if you for arn:aws:ssm:us-east-2:[my account number]:parameter/Dev/WebServer/[thing I want to get] make [thing I want to get] something like myapp-prefix-[thing I want to get] then you can easily limit the access only to parameters which belongs to this given app. Or you can use tags.
botocore.exceptions.ClientError:
An error occurred (AccessDeniedException) when calling the CreateFunction operation: Lambda was unable to encrypt your environment variables because the KMS access was denied. Please check your KMS permissions.
KMS Exception: AccessDeniedException KMS Message:
User: arn:aws:iam::816087693535:user/pj#123.com is not authorized to perform: kms:Encrypt on resource: arn:aws:kms:us-east-1:816087693535:key/22a02ce6-2afe-4184-bfbf-18d72d238bdd with an explicit deny in a service control policy.
Also we tried to read and write kms policies but it did not work out.
You need to allow the IAM user pj#123.com to perform kms:Encrypt on the resource arn:aws:kms:us-east-1:816087693535:key/22a02ce6-2afe-4184-bfbf-18d72d238bdd If you have already tried that but it still doesn't work my next question would be if you are using AWS Organizations. If you do, I would visit AWS Organizations on the Management Account and check for any Service Control Policies (SCPs) which explicitly deny this action. If you follow these steps the resolution of your issue should be straightforward. Kindly let me know if not.
This stack was working at one point... I'm not sure what's going on. This permission is no longer doing what it did before, or has become invalid.
I have a Lambda function that rotates a Secret, so naturally it must be triggered by Secrets Manager. So I built up the Permission as follows
import * as aws from '#pulumi/aws'
export const accessTokenSecret = new aws.secretsmanager.Secret('accessTokenSecret', {});
export const smPermission = new aws.lambda.Permission(`${lambdaName}SecretsManagerPermission`, {
action: 'lambda:InvokeFunction',
function: rotateKnacklyAccessTokenLambda.name,
principal: 'secretsmanager.amazonaws.com',
sourceArn: accessTokenSecret.arn,
})
And the Policy,
{
Action: [
'secretsmanager:GetResourcePolicy',
'secretsmanager:GetSecretValue',
'secretsmanager:DescribeSecret',
'secretsmanager:ListSecrets',
'secretsmanager:RotateSecret',
],
Resource: 'arn:aws:secretsmanager:*:*:*',
Effect: 'Allow',
},
Running pulumi up -y yields
aws:secretsmanager:SecretRotation (knacklyAccessTokenRotation):
error: 1 error occurred:
* error enabling Secrets Manager Secret "" rotation: AccessDeniedException: Secrets Manager cannot invoke the specified Lambda function. Ensure that the function policy grants access to the principal secretsmanager.amazonaws.com.
This error confuses me, because the Policy created for the Lambda will not accept the Principal param (which makes sense, the same behaviour happens in the AWS Console), so I'm sure they mean Permission instead of Policy.
Based on the log I can tell that the Permission is being created way after the Lambda/Secrets Manager is, I'm not sure if this is a Pulumi issue similar to how it destroys stacks in the incorrect order (Roles and Policies for example).
I can see the Permission in the AWS Lambda configuration section, so maybe it's ok?
I want to grant vpc access for my lambda function. I use the following aws cli command.
aws lambda update-function-configuration \
--function-name SampleFunction \
--vpc-config SubnetIds=subnet-xxxx,SecurityGroupIds=sg-xxxx
But I receive the following error:
An error occurred (AccessDeniedException) when calling the
UpdateFunctionConfiguration operation: Your access has been denied by
EC2, please make sure your request credentials have permission to
DescribeSecurityGroups for sg-xxxx. EC2 Error Code:
UnauthorizedOperation. EC2 Error Message: You are not authorized to
perform this operation.
I have granted the following permission to both my lambda role and the user who execute the aws command.
- "ec2:CreateNetworkInterface"
- "ec2:DescribeNetworkInterfaces"
- "ec2:DeleteNetworkInterface"
- "ec2:DescribeSecurityGroups"
I further tried to grant full access to both the lambda role and the user. But still received the same error
Can anyone suggest what else I can try?
The trick is to add the pipeline / worker role / user which is deploying the lambda function) have access to network related policies. The lambda function should itself suffice with managed policy - AWSLambdaVPCAccessExecutionRole
arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole
Action:
ec2:DescribeSecurityGroups
ec2:DescribeSubnets
ec2:DescribeVpcs
Effect: Allow
Resource: '*'
Your users IAM policy needs further permissions.
For example ec2:CreateSecurityGroup & etc. Have a look at this documentation to add requred permissions.
I experienced the same issue. Despite the IAM policy for the user having the required permissions, I could not use the aws cli to crate a lambda function with a VPC config (aws lambda create-function) or modify an existing function to add a VPC config (aws lambda update-function-configuration).
The only way I could get this to work was to create the lambda function without a VPC config. I then modified the function to add the VPC config information (vpc, subnet and security groups) via the AWS console (in Lambda > Fucntions > My Function > Network). I was only able to use the console to do this, introducing a manual step in an otherwise fully automated process.
To answer some of the questions above about which user needs the ec2:DescribeSecurityGroups and related permissions. It is the user running the cli command or logged in to the console. The function does not need a policy providing these permissions. The only special permissions needed for a function with a VPC config are:
ec2:CreateNetworkInterface
ec2:DescribeNetworkInterfaces
ec2:DeleteNetworkInterface
These allow the function to create ENIs within your VPC using the subnet and security group you provide as described here.
Both the Lambda funtion's role and the user role (either cloudformation or cmline user) must have:
- ec2:CreateNetworkInterface
- ec2:DescribeNetworkInterfaces
- ec2:DeleteNetworkInterface
- ec2:DescribeSecurityGroups
- ec2:DescribeSubnets
or ec2:* if ok for your use case'security
I had the same issue deploying a lambda with a VPC config using SAM/cloudformation and resolved it by adding this above.
on github issue some people say it is because of cloudformation order creation it is not (or maybe not anymore because I tested adding 20 dummy resource and still the same issue only resolved by adding the permissions above)
cheers,