I was wondering if there are any migration guides for migrating from GKE Secrets to the Secret Manager API?
I'm not aware of any existing guides for doing this.
Basically, you'll need to
recreate all of your secrets in Secret Manager
modify your application code to use the Secrets Manager API to access the secrets
ensure you have Workload Identity enabled for your cluster
make sure your pod(s) use a identity with access to Secrets Manager.
assuming everything above works, delete your existing Secrets
By default, data in GKE secrets is stored in Base64 encoding, which is practically the same as plaintext. GCP Secret Manager are encrypted with Google-default encryption. With Google-default encryption, secret payloads are encrypted by keys managed by Google.
See this documentation.
Related
https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/
According to the above document in order to use encryption configuration, we need to edit the kube-apiserver.yaml file. But in GCP, Azure or AWS we cannot view this the api-server as it is managed by the cloud provider. How can we use encryption configuration in this case? Has anyone managed to use encryption configuration to encrypt secrets in GCP,Azure and AWS?
Google Secret Manager(GSM)is GCP’s flagship service for storing, rotation and retrieving secrets. A secret in GSM could be stored in encrypted form. It supports IAM for authentication and fine grained access controls
Azure Key Vault FlexVolume and for aws Amazon Elastic Container Service for Kubernetes (EKS) are the other tools that can be used
I am trying to figure out is there a way to access secrets like db password from AWS Batch job using Vault/ K8S secrets. The option to use Secret Manager provided by AWS is there, but as per my company policy we are using vault across the projects. AWS batch documentation does not have any documentation on options to store secrets other than Secret Manager.
Can anyone help me if using vault/ k8s secret is an option with AWS Batch.
How in a work environment with different AWS environments say for example develop, staging and production is it best to store the AWS Access Key and Secret Key other than in the appsettings.json files in .Net Core? I know there is Secret Manager but not sure if that is the best way to store these two values. Looking for someone that may have done this specifically for production and how they handled this within their organization. Thanks for any information.
I believe that your application is running outside of AWS and that it needs to make API calls to AWS services, for example SQS. To make those API calls, your application needs AWS credentials.
Here are approaches for authenticating external applications in a machine-to-machine scenario. In your case, your client seems to need to be able to make arbitrary AWS service requests and that means using AWS signature v4 requests, signed using AWS credentials, which are ideally temporary, rotated credentials from STS rather than persistent credentials (such as IAM user credentials).
Typically, you would configure your application with a base set of IAM credentials that allow the application to assume an IAM role. That role itself, rather than the base credentials, would then give your application the permissions it needs to make SQS API calls etc.
The issue you face is how to securely store the base set of credentials. This is a problem that on-premise applications have had since day one, well before the cloud era, and there are various solutions, depending on the technology you're using.
Typically these credentials would be encrypted, not committed to code repos, and populated on the relevant, locked down application servers in some secure fashion. Some potentially useful resources:
Encrypting sections of a configuration file for an ASP.NET application
Use AWS Secrets Manager to store & read passwords in .Net Core apps
Securely store and retrieve sensitive info in .NET Core apps with Azure Key Vault
AWS Secret Manager securely stores your secrets until you retrieve them at runtime. If your going to be running your ASP.NET Core app in AWS, then AWS Secrets Manager is a great option, as it allows you to finely control the permissions associated with the AWS IAM roles running your apps.
Here are some faqs which were given from the AWS for secrets-manager service and which will clear your doubts also.
Here is the article which you can refer to for implementing secure secrets storage for .net core with AWS Secret Manager
Does anyone know in which case choose Kubernetes secrets instead of google secret manager and the reverse ? Differences between the two ?
With Kubernetes secret (K8S Secret), you use a built in feature of K8S. You load your secrets in config maps, and you mount them on the pods that require them.
PRO
If a day you want to deploy on AWS, Azure or on prem, still on K8S, the behavior will be the same, no update to perform in your code.
CONS
The secrets are only accessible by K8S cluster, impossible to reuse them with another GCP services
Note: With GKE, no problem the ETCD component is automatically encrypted with a key form KMS service to keep the secret encrypted at rest. But, it's not always the same for every K8S installation, especially on premise, where the secrets are kept in plain text. Be aware about this part of the security.
Secret Manager is a vault managed by Google. You have API to read and write them and the IAM service checks the authorization.
PRO
It's a Google Cloud service and you can access it from any GCP services (Compute Engine, Cloud Run, App Engine, Cloud Functions, GKE,....) as long as you are authorized for
CONS
It's Google Cloud specific product, you are locked in.
You can use them together via this sync service: https://external-secrets.io/
Is there a way to retrieve a secret from the AWS secret store using DefaultAWSCredentialsProviderChain java class?
If not please suggest a way to retrieve it? (I need this in the context of doing signature V4 signing the request to connect with AWS Neptune. For signature signing, I am using this example. But my secrets are in AWS secret manager, So How can I retrieve the secret from the secret store with DefaultAWSCredentialsProviderChain)
I'm not sure if Secrets Manager exposes a AWSCredentialsProvider interface, but even if they don't support one, it should be easy to write something up.
Here is a sample implementation that uses the Secrets Manager APIs to expose a credentials provider implmentation:
https://github.com/jenkinsci/aws-secrets-manager-credentials-provider-plugin/blob/0e12e02a759d13524ed7f5cd0125ef6eab47ff7d/src/main/java/io/jenkins/plugins/credentials/secretsmanager/AwsCredentialsProvider.java
Once you have something like this, just make sure you pass it to the SigV4Signer that you use in your application.
Reference: https://github.com/aws/amazon-neptune-sigv4-signer/blob/master/src/main/java/com/amazonaws/neptune/auth/NeptuneSigV4SignerBase.java#L77-L86
Hope this helps.
It is possible with aws secretsmanager
Use these docs