I'm using AWS Fargate and storing sensitive data with Secrets Manager. Task definition should get environment variables from secrets store
- name: "app"
image: "ecr-image:tag"
essential: true
secrets:
- name: "VAR1"
valueFrom: "arn:aws:secretsmanager:us-east-1:111222333444:secret:var-one-secret"
- name: "VAR2"
valueFrom: "arn:aws:secretsmanager:us-east-1:111222333444:secret:var-two-secret"
- name: "VAR3"
valueFrom: "arn:aws:secretsmanager:us-east-1:111222333444:secret:var-two-private"
but for some reason it fails with the error below
ResourceNotFoundException: Secrets Manager can’t find the specified secret. status code: 400, request id
It seems a bit strange to me because
IAM has permissions for get secret value, moreover
when leaving only VAR1 variable everything works as expected
AWS CLI is able to retrieve each secret without any issue
e.g.
aws secretsmanager get-secret-value --secret-id var-two-secret
What might be wrong with my configuration? Any hints appreciated
ok, so the trick was to specify ARN explicitly. Instead of just providing secret name you should use full identifier
arn:aws:secretsmanager:us-east-1:111222333444:secret:var-two-secret-ID0o2R
Note -ID0o2R suffix at the end of secret name.
It's still not clear for me why for some variables it works without it.
UPD
However, if your secret has a name that ends in a hyphen followed by
six characters (before Secrets Manager adds the hyphen and six
characters to the ARN) and you try to use that as a partial ARN, then
those characters cause Secrets Manager to assume that you’re
specifying a complete ARN. This confusion can cause unexpected
results.
So as you can see from my variable name containing a hyphen Secrets Manager had hard times when resolving it by short name
Secrets Manager tries to do partial ARN matching when you do not specify the GUID on the end of the ARN. However, it is imperfect because partial ARNs could collide. If you are fetching secrets within the same account, you can just use the secret name (the part after secret: and excluding the dash 6 character -GUID) instead of the full ARN. But using the full ARN, when you have it, is always best.
Another potential cause of this error is that the secret isn’t set; the secret name might exist, but not have a value. See https://docs.aws.amazon.com/secretsmanager/latest/userguide/manage_update-secret.html for steps on setting a value.
Just add a double colon to the end of the ARN:
"arn:aws:secretsmanager:us-east-1:1234567890:secret:example-ABC12:VARIABLE_NAME::"
Explanation:
arn:aws:secretsmanager:us-east-1:1234567890:secret:example-ABC12 is
the ARN of your secrets (vault)
VARIABLE_NAME is the actual variable you added, with the addition of :: to the ARN.
Check all the possible combinations in the docs.
Related
I'm using Terraform to deploy an ECS cluster, and task definition has these sections:
{
name = "secret"
valueFrom = "SECRET_NAME"
}
Now, if I'm using a parameter, I can just place a static parameter ARN in the valueFrom field. However, I'm using secrets and Secrets Manager. AWS automatically adds 6 random characters to every deployed secret ARN. I have 4 environments, all taken care of by a single pipeline. I obviously need a way to fit all 4 environments and their secrets in a single valueFrom field. Does this field only take an ARN, or is there some other way for me to do this?
Edit:
I forgot to mention I already have these secrets as an imported resource by name:
data "aws_secretsmanager_secret" "secret" {
name = "secret_name"
}
Is there something I can do with that?
I am very new to Kubernetes. My task is to move the existing application from Kubernetes to EKS. I am using CDK EKS Blueprints to create the cluster in AWS and have AWS secret manager to create the Kubernetes secret. I followed the same steps as given in here https://aws-quickstart.github.io/cdk-eks-blueprints/addons/secrets-store/
As mentioned on the above page I got the service account, a role in the service account to access the secret and the secret created.
Though I have a volume block, mount path for the secret and used env variables to refer the secret, I am not able to get my pod up and running. Instead it complains that the key is not found in the secret.
The reason may be because when I try to create a secret manually using the create command the Kubernetes create the secret as below.
enter image description here
But when the Kubernetes secret is created by EKS blueprints by lookingup the existing AWS secret like
secretProvider: new blueprints.LookupSecretsManagerSecretByName('test-aws-secret'),
it is creating as an encoded object.
enter image description here
Now I am not sure how to reference the nested object in the yaml. I tried many iterations, something like enter image description here. But no luck. Any help is much appreciated.
Thanks.
The value of the key field should be key1:
- name: key1-value
valueFrom:
secretKeyRef:
name: secret-test
key: key1
Including data/secret-test/ before the key name is unnecessary because Kubernetes already knows the secret name from the name field and knows to look for keys under the data field of secrets.
See Secrets for more information.
I am planning to use AWS parameter store to store config for one of the project I am working on it. We are using cloud formation (or CDK) to deploy all the components. That includes parameter store as well.
I have some config which has password and other sensitive fields which I can't put to in version control. How to handle this scenario?
I would use AWS Secrets Manager to generate the secrets randomly.
#This is a Secret resource with a randomly generated password in its SecretString JSON.
MyRDSInstanceRotationSecret:
Type: AWS::SecretsManager::Secret
Properties:
Description: 'This is my rds instance secret'
GenerateSecretString:
SecretStringTemplate: '{"username": "admin"}'
GenerateStringKey: 'password'
PasswordLength: 16
ExcludeCharacters: '"#/\'
Tags:
-
Key: AppName
Value: MyApp
And would further export the same into AWS Parameter Store using a policy attached and later on access them using static or dyanmic reference.
The best would be to take your secrets management out of Cloudformation as suggested by #jordanm.
Take a look at AWS Secrets Manage for this use case. If you are implementing your solution in Java, see this Github URL:
https://github.com/awsdocs/aws-doc-sdk-examples/tree/master/javav2/example_code/secretsmanager
Provisioning SecureString parameter type is not possible in clouldforamtion
AWS CloudFormation doesn't support creating a SecureString parameter
type
See the following link: This
But you can reference it securely, using dynamic references which provide a compact, powerful way for you to specify external values that are stored and managed in other services, such as the Systems Manager Parameter Store, in your stack template.
Use the ssm-secure dynamic reference pattern to specify AWS Systems
Manager SecureString type parameters in your templates. For ssm-secure
dynamic references, AWS CloudFormation never stores the actual
parameter value. AWS CloudFormation accesses the parameter value
during create and update operations for stacks and change sets.
Check the following link:This
I am trying to fetch pre existing secrets from the aws-secretsmanager module on CDK, and from the documentation here, the suggestion is
If you need to use a pre-existing secret, the recommended way is to
manually provision the secret in AWS SecretsManager and use the
Secret.fromSecretArn or Secret.fromSecretAttributes method to make it
available in your CDK Application
However, both the methods demand the use of the arn to fetch the secrets. I am not sure if it is a good idea to hardcode arns and check them into the git repo. Instead is there a way to just fetch the secrets by just using the name, since we already have the account details available in the profile for cdk.
At least until this current version (1.38.0), it’s not possible. An alternative is to save the secret arn in the SSM parameter store and use the ssm key in the code.
Putting full ARNs in CFN should not be a concern. Since you are creating these secrets ahead of time, their name, account, and region will be know. If you wish, however, you could still use the CFN psuedo parameters for partition, region, and account (AWS::Partition, AWS::Region, AWS::AccountId or the CDK equivelent).
I currently have the following problem. I am creating a Template in which I specify a ServiceAccount adn a RoleBinding. Openshift Creates a Token on its own and stores it in a secret with the name [service-account-name]-[a-z,1-9{5}]. Now I want to pass that secret on to an env Variable (as it will be consumed by another config in that container that can process env variables)
Now you can easily use env variables like
env:
- name: something
valueFrom:
secretKeyRef:
name: someKey
key: someValue
But now I've got the problem, that there is a secret, but I don't know the exact name as part of it is random. Now my question is
Is there a way to use the contents of a secret of a serviceaccount in a template?
You can check your secrets by running
kubectl get secret and then view more by running kubectl describe secret mysecret You will need to decode it to view it (I do not have experience with OpenShift). You can also use them as Environment Variables as explained here.
As for ServiceAccount and the token you can use it inside a container as specified in the OpenShift documentation
A file containing an API token for a pod’s service account is
automatically mounted at
/var/run/secrets/kubernetes.io/serviceaccount/token.
I think you could add commands from the documentation to the Pod Template into command: section similar to this example. Also you can find more about using secrets here.