https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/
According to the above document in order to use encryption configuration, we need to edit the kube-apiserver.yaml file. But in GCP, Azure or AWS we cannot view this the api-server as it is managed by the cloud provider. How can we use encryption configuration in this case? Has anyone managed to use encryption configuration to encrypt secrets in GCP,Azure and AWS?
Google Secret Manager(GSM)is GCP’s flagship service for storing, rotation and retrieving secrets. A secret in GSM could be stored in encrypted form. It supports IAM for authentication and fine grained access controls
Azure Key Vault FlexVolume and for aws Amazon Elastic Container Service for Kubernetes (EKS) are the other tools that can be used
Related
I was going by this update for EKS https://aws.amazon.com/about-aws/whats-new/2020/03/amazon-eks-adds-envelope-encryption-for-secrets-with-aws-kms/ and this blog from AWS https://aws.amazon.com/blogs/containers/using-eks-encryption-provider-support-for-defense-in-depth/.
This is a very cryptic line which never confirms whether EKS encrypts secrets or not by default
In EKS, we operate the etcd volumes encrypted at disk-level using AWS-managed encryption keys.
I did understand that:-
KMS with EKS will provide envelope encryption,like encrypting the DEK using CMK.
But it never mentioned that if I don't use this feature ( of course KMS will cost ), does EKS encrypts data by default?
Because Kubernetes by default does not encrypt data . Source
Kubernetes Secrets are, by default, stored unencrypted in the API server's underlying data store (etcd). Anyone with API access can retrieve or modify a Secret, and so can anyone with access to etcd. Additionally, anyone who is authorized to create a Pod in a namespace can use that access to read any Secret in that namespace; this includes indirect access such as the ability to create a Deployment.
I think I found it, the blog and update post by aws are very cryptic.
According to docs and console :-
All of the data stored by the etcd nodes and associated Amazon EBS volumes is encrypted using AWS KMS.
Using KMS with EKS is additional encryption or a better way of envelope encryption. It allows deploying a defense-in-depth strategy for Kubernetes applications by encrypting Kubernetes secrets with a KMS key that you define and manage.
short answer, yes it encrypted at rest
The answer is yes, the data stored by etcd is encrypted at rest.
Encrypt secrets at rest in etcd
This encryption is in addition to the EBS volume encryption that is enabled by default for all data (including secrets) that is stored in etcd as part of an EKS cluster. so When encryption is enabled, then the secret store is encrypted form using KMS within etcd
The info tab contains further information
AWS EKS is a managed Kubernetes offering. Kubernetes control plane components such as API Server, and ETCD are installed, managed, and upgraded by AWS. Hence you can neither see these components nor can exec into these components.
The below article also shows how to get cloud trail events when Kubernetes secrets are decrypted using KMS.
eks-encryption
Ensure AWS EKS cluster has secrets encryption enabled
I was wondering if there are any migration guides for migrating from GKE Secrets to the Secret Manager API?
I'm not aware of any existing guides for doing this.
Basically, you'll need to
recreate all of your secrets in Secret Manager
modify your application code to use the Secrets Manager API to access the secrets
ensure you have Workload Identity enabled for your cluster
make sure your pod(s) use a identity with access to Secrets Manager.
assuming everything above works, delete your existing Secrets
By default, data in GKE secrets is stored in Base64 encoding, which is practically the same as plaintext. GCP Secret Manager are encrypted with Google-default encryption. With Google-default encryption, secret payloads are encrypted by keys managed by Google.
See this documentation.
Fluxcd ImageRepository authentication with AWS Elastic Container Registry Not working on ARM64 graviton node.
After debugging I found that the image used in the init container to get cred credentials is not supporting Arm64 instances.
image name:-bitnami/kubectl
doc link:-https://fluxcd.io/docs/guides/image-update/
There are some workarounds, provided on the fluxcd documentation portal:
AWS Elastic Container Registry
Using a JSON key
Using Static Credentials
AWS Elastic Container Registry
The solution proposed is to create a cronjob that runs every 6 hours which would re-create the docker-registry secret using a new token.
JSON key
A Json key doesn’t expire, so we don’t need a cronjob, we just need to create the secret and reference it in the ImagePolicy.
First, create a json key file by following this documentation. Grant the service account the role of Container Registry Service Agent so that it can access GCR and download the json file.
Static Credentials
Instead of creating the Secret directly into your Kubernetes cluster, encrypt it using Mozilla SOPS or Sealed Secrets, then commit and push the encypted file to git.
This Secret should be in the same Namespace as your flux ImageRepository object. Update the ImageRepository.spec.secretRef to point to it.
I am trying to understand the key management services in AWS (Amazon Web Services) and I can see that Amazon recommends more AWS Key Management Service (KMS) over Cloud Hardware Security Module (Cloud HSM). But I am having a hard time finding the key differences between the two, KMS vs Cloud-HSM.
Can someone please list a few key differences or a comparison of the two technologies?
Feature
AWS Cloud HSM
AWS KMS
Tenancy
Single-Tenant
Multi-Tenant
High Availability: How to achieve?
Create multiple HSMs (manually) over different AZs
Managed (automatically) by AWS
Scaling/Performance Responsibility
Your responsibility
AWS
Key access: Who controls it?
You
You+AWS
Keys: How to use?
Customer code + Safenet APIs
AWS Management Console
Keys: Where to use?
AWS & Your Network (VPN)
AWS
AWS Services Integration
A small set of services (Redshift, Oracle RDS etc.)
Most services fully integrated
Access & Authentication Policy
Quorom based K of N
AWS IAM Policy
Price
$$
$
FIPS 140-2 Compliance
Level 3
Level 2 overall (Level 3 in some areas)
Source: AWS official documentation + multiple courses I took for the AWS exams + practical experience.
Developers describe AWS CloudHSM as "Dedicated Hardware Security Module (HSM) appliances within the AWS cloud". The AWS CloudHSM service allows you to protect your encryption keys within HSMs designed and validated to government standards for secure key management. You can securely generate, store, and manage the cryptographic keys used for data encryption such that they are accessible only by you. AWS CloudHSM helps you comply with strict key management requirements without sacrificing application performance.
On the other hand, AWS Key Management Service is detailed as "Easily create and control the encryption keys used to encrypt your data".
AWS Key Management Service (KMS) is a managed service that makes it easy for you to create and control the encryption keys used to encrypt your data, and uses Hardware Security Modules (HSMs) to protect the security of your keys. AWS Key Management Service is integrated with other AWS services including Amazon EBS, Amazon S3, and Amazon Redshift. AWS Key Management Service is also integrated with AWS CloudTrail to provide you with logs of all key usage to help meet your regulatory and compliance needs.
AWS CloudHSM and AWS Key Management Service can be categorized as "Data Security Services" tools.
Some of the features offered by AWS CloudHSM are:
1]Protect and store your cryptographic keys with industry standard, tamper-resistant HSM appliances. No one but you has access to your keys (including Amazon administrators who manage and maintain the appliance).
2]Use your most sensitive and regulated data on Amazon EC2 without giving applications direct access to your data's encryption keys.
3]Store and access data reliably from your applications that demand highly available and durable key storage and cryptographic operations.
On the other hand, AWS Key Management Service provides the following key features:
1]Centralized Key Management
2]Integrated with AWS services
3]Encryption for all your applications
The AWS KMS custom key store feature combines the controls provided by AWS CloudHSM with the integration and ease of use of AWS KMS
Q: Why would I need to use a custom key store?
Since you control your AWS CloudHSM cluster, you have the option to manage the lifecycle of your CMKs independently of AWS KMS
From official documentation, it seems that KMS is a basic feature, and you can get a senior feature by expanding with CloudHSM.
Does anyone know in which case choose Kubernetes secrets instead of google secret manager and the reverse ? Differences between the two ?
With Kubernetes secret (K8S Secret), you use a built in feature of K8S. You load your secrets in config maps, and you mount them on the pods that require them.
PRO
If a day you want to deploy on AWS, Azure or on prem, still on K8S, the behavior will be the same, no update to perform in your code.
CONS
The secrets are only accessible by K8S cluster, impossible to reuse them with another GCP services
Note: With GKE, no problem the ETCD component is automatically encrypted with a key form KMS service to keep the secret encrypted at rest. But, it's not always the same for every K8S installation, especially on premise, where the secrets are kept in plain text. Be aware about this part of the security.
Secret Manager is a vault managed by Google. You have API to read and write them and the IAM service checks the authorization.
PRO
It's a Google Cloud service and you can access it from any GCP services (Compute Engine, Cloud Run, App Engine, Cloud Functions, GKE,....) as long as you are authorized for
CONS
It's Google Cloud specific product, you are locked in.
You can use them together via this sync service: https://external-secrets.io/