This question is about the rotation of AWS secrets manager. When creating a secret, you could choose rotation frequency, but I can't figure out how rotation works.
Imagine a scenario like the following.
You create a secret A1 in aws secret manager, and specify the rotation frequency is 30 days.
You encrypt the data with A1.
30 days later, A1 has been rotated to be A2.
then, you program retrieves AWS secret manager and got the value of A2. How do you decrypt the data that has been encrypted with A1?
Secrets Manager rotation is primarily used for API keys or passwords.
Your encrypted values are stored in the Secrets Manager secret, but the encryption key itself is stored in KMS.
When the rotation occurs, these values are replaced by the same KMS customer master key will be used to encrypt the new value.
If you're ever wanting to store an encryption key you would use either AWS KMS, or AWS CloudHSM (if your organisation has specific regulatory requirements or wants to invest in a dedicated HSM).
You should use KMS for storing encryption keys, not Secrets Manager. You can rotate encryption keys in KMS while keeping the old keys available.
Secrets Manager is for things like passwords, where rotation involves updating an account to have a new password, and the old password would be no longer needed.
Related
How exactly does encryption key rotation work? I understand it's a very good practice to continuously rotate your encryption keys for security purposes, but rotating a key would require too much work.
Case:
Let's just say I have a database storing 30GB of data, and we're using an internal key to encrypt data at rest, and I plan to rotate my keys every month.
Questions:
Does that mean all my data will be decrypted by the old key and re-encrypted by the new one every month?
The whole encryption-decryption would take a lot of time and compute resources.
If my DB ( or any encrypted dataset ) scales tomorrow, does that mean the same process would duplicate when my key rotates? This does not look like a scalable solution.
Other Details:
I've also seen AWS KMS rotates it's keys if we've selected the rotation option. How does AWS manage to rotate it's keys and all encrypted data for all the underlying services?
You need to familiarize yourself with Envelope Encryption. Each time you want to encrypt data in AWS, you first generate a unique data-key. You then encrypt your data with this key. This key is not the key that is rotated!
Then you take this key, and you encrypt it with a key from KMS. Now if you want to decrypt this data, you must first get the decrypted data key, and to decrypt this data key, you will need the KMS key.
Now if you want to rotate the key, you don't need to re-encrypt all the data, instead you need to decrypt the data key using your key to be rotated from KMS, and then get a new key, and re-encrypt the unencrypted data key. That way you don't need to re-encrypt all the data.
Here are two important links that can help you understand envelop encryption and key rotation in AWS.
https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html
https://docs.aws.amazon.com/kms/latest/developerguide/rotate-keys.html
I have quoted some important concepts from the above links:
Envelop encryption:
When you encrypt your data, your data is protected, but you have to protect your encryption key. One strategy is to encrypt it. Envelope encryption is the practice of encrypting plaintext data with a data key, and then encrypting the data key under another key.
Customer Master Keys
CMKs are created in AWS KMS. Symmetric CMKs and the private keys of asymmetric CMKs never leave AWS KMS unencrypted. This strategy differs from data keys. AWS KMS does not store, manage, or track your data keys. You must use them outside of AWS KMS.
Data Keys
Data keys are encryption keys that you can use to encrypt data, including large amounts of data and other data encryption keys. You can use AWS KMS customer master keys (CMKs) to generate, encrypt, and decrypt data keys. However, AWS KMS does not store, manage, or track your data keys, or perform cryptographic operations with data keys. You must use and manage data keys outside of AWS KMS.
Key rotation
When you enable automatic key rotation for a customer managed CMK, AWS KMS generates new cryptographic material for the CMK every year. AWS KMS also saves the CMK's older cryptographic material in perpetuity so it can be used to decrypt data that it encrypted. AWS KMS does not delete any rotated key material until you delete the CMK.
An important concept in Key rotation is the HSM backing key(HBK):
(https://docs.aws.amazon.com/kms/latest/cryptographic-details/key-hierarchy.html)
Within the hierarchy of a specific CMK, the HBK can be thought of as a version of the CMK. When you want to rotate the CMK through AWS KMS, a new HBK is created and associated with the CMK as the active HBK for the CMK. The older HBKs are preserved and can be used to decrypt and verify previously protected data. But only the active cryptographic key can be used to protect new information.
Does that mean all my data will be decrypted by the old key and re-encrypted by the new one every month?
As already answered, the simple answer is no. But the previous answers miss the purpose of the key rotation
The reason behind the key rotation is limit amout of data encrypted by a single key.
How does AWS manage to rotate it's keys and all encrypted data for all the underlying services?
The basic idea (at least in KMS) is, that the KMS key is not a single key, but it is a set of keys, which the last one is the current one. You can imagine that as "key versioning". After each key rotation the current key is saved so you can still decrypt the previously encrypted ciphertext (data key - as mentioned in other answers). I believe in the KMS this whole set is hidden, but Azure KeyValt shows the whole set as key version.
The whole encryption-decryption would take a lot of time and compute resources
No, you don't need to do anything. AWS manages the "key versions" for you. Even if you'd re-encrypt the envelope keys as suggested in another answer, then you would actually fail to fulfill the purpose of the key rotation.
The ciphertext generated by the KSM must contain some identification of the key version too, so the KMS is able to decrypt the ciphertext even after the key is rotated.
I'm implementing a service that requires me to call my customers' API using their API keys. My customers will provide me with their API keys in their accounts.
When I'm calling my customers' API, I have to retrieve their API key before making the call. Since these are my customers' API keys and I want them to be kept safely, I'm considering keeping all of them in AWS Secrets Manager. I have roughly about 5,000 users (still growing) and I plan to store all their keys into a single secret in Secrets Manager. My application makes about a few millions calls to my customers API a month and it needs to retrieve the keys at high frequency and concurrency.
However, I'm not sure if this is the kind of use case for Secrets Manager because their docs sound to me like it was meant for just keeping secret information for the application and not for customers like a database. At the same time, storing encrypted keys in the database and having to decrypt them with a KMS key sounds like I may end up with roughly the same cost.
Is Secrets Manager meant for such a use case to store customers' sensitive information such as API keys? If not, what should I consider in my case?
50k api keys in a single secret is goinfg to be very unwieldy. Assuming a 40 byte token, you're looking at 2mb of data - SSM has a max data length for a value of 4096 bytes unless I'm mistaken.
To me it would make more sense to generate a key with KMS and use that key to encrypt customer API keys before writing them to a DynamoDB table (or even RDS if you so desire) When you need to use a customer API key, fetch it from dynamoDB, decrypt it with the KMS key, and then make use of it.
If you want automatic key rotation, SSM could be used to encrypt the key you use to encrypt the client API tokens. Your token decryption key would remain usable while the wrapping SSM entry would be reencrypted with a key rotation set by policy.
Finally, as Software Engineer suggested above, there is Vault.
I am wondering if you please help me out with the following question.
What are the differences between the KMS and the secret manager in GCP? Thank you in advance.
https://cloud.google.com/secret-manager/docs/
HB
Cloud KMS encrypts data and returns the encrypted ciphertext. Cloud KMS does not store the secret, only the keys to encrypt/decrypt.
Secret Manager actually stores the secret material. Secret Manager also keeps a history (versions) of secret material. All data in Secret Manager is encrypted. By default, it is encrypted with a Google-managed key. You can actually use Cloud KMS to encrypt Secret Manager secrets (this is called "CMEK"), in which case the user controls the keys.
Cloud KMS is designed as a cryptographic oracle system: nobody, including yourself, can get the keys out: this means they're locked inside the system and you don't have to worry in practice about them leaking. The tradeoff is that the only thing you can do with those keys is encrypt, decrypt, and other cryptographic operations: useful for protecting data, or even for encrypting secrets, but if you have a database password or something else which you want to keep secret, but then actually be able to use or send elsewhere, you have to store the encrypted version, then use Cloud KMS to decrypt it.
When you do have configuration info like a database password, where your software actually needs the secret, not cryptographic operations, then Secret Manager is designed for that use case. The tradeoff is that if you get a copy of the secret out, it's harder to keep it from leaking and be certain it's controlled.
Thanks for using GCP!
I found that AWS RDS allows encrypting DB resources with AWS KMS. Because it is done inside the AWS infrastructure the encryption key can be easily rotated automatically. It is cool, but it is only encryption-at-rest.
I would additionally like to have encrypted some particular columns in the database. For example SSN. I would like to store them encrypted and decrypt them to display inside my application. Moreover, I would like to have an individual key for every user.
The main problem which I observed will be the rotation of the key. As I'm thinking to rotate the key for one user I would like to do this inside my application:
get a current encryption key from KMS
decrypt all the data from RDS encrypted with the current key
generate a new encryption key
encrypt everything again and store data in RDS
store the new key in the KMS
The main problem here would be to keep everything in a "transaction" - to "commit" if everything was fine and to "rollback" everything if anything went wrong.
I wonder if such keys rotation for the encryption at the columns level could be done inside the AWS infrastructure automatically. Do you have any ideas about that? Maybe you know any other, better approach for such a situation?
What problem are you solving by having individual keys per user? The KMS paradigm is to use policy to grant access to a Customer Master Key (CMK). As Mark pointed out above, there is a limit on the number of keys.
Have a look at this walkthrough
There is a section at the bottom about Key rotation strategies that might help:
"A recommended approach to manual key rotation is to use key aliases within AWS KMS. This allows users to always select the same key alias when configuring databases, while the key administrator rotates the underlying CMK. By keeping the old CMK, you allow any applications that currently use this key to still decrypt any data that was encrypted by it, as long as the CMK key policy still gives the AWSServiceRoleForRDS role permission as a Key User. It also allows for any new data to be encrypted with the new CMK."
I would additionally like to have encrypted some particular columns in the database.
Then I assume you will use a form of key wrapping ( data keys )
The main problem which I observed will be the rotation of the key
..
encrypt everything again and store data in RDS
store the new key in the KMS
The purpose of the key rotation is limit amount of data encrypted by a single key, not to re-encrypt the whole encrypted content.
Suggestion:
as already linked - encrypt your data using a random (per row?) data key.
encrypt the data key using the user-specific key.
encrypt the user-key with a KMS-key
Key Rotation (KMS) should be transparent for you with no action to do.
If you wish to manually rotate the user keys, you may, IMHO it is not so critical - the user keys are used to encrypt the data keys - relatively short data with high entropy.
IMHO the action by rotation taken should be creating a new key and preserving the old one for decryption purposes, not to re-encrypt all the database records
Are you using something like AWS Cognito to give each user temporary IAM permissions when they log in? Otherwise, I don't see the point of giving each user a separate KMS key, since you would be giving a single entity (your server) access to all the keys. Also note that there is a limit of 10,000 KMS keys in an AWS account. That's quite a bit, but if you have a website where anybody can sign up you might run up against that limit if you are assigning each user a key.
As for handling key rotation automatically, since you have to write the custom code to encrypt individual column values in your database, you will also have to write the code to perform all the steps of the key rotation process. You could create that as an AWS Lambda function and configure it to run "automatically", but it isn't something Amazon gives you out of the box without any custom code.
I need to develop a solution to store both symmetric and asymmetric keys securely in AWS. These keys will be used by applications that are running on EC2s and Lambdas. The applications will need to be set up with policies that will allow the application or lambda to pull the keys out of the key store. The key store should also manage the key expiry, notifying various people when keys are going to expire. The initial key exchange is between my company and its partners meaning that we may have either a public or private key for a key pair depending upon the data transfer direction.
We have looked at KMS but from what I have seen KMS does not support asymmetric keys. I have also seen online that some people are using either S3 (protected by KMS) or parameter store to store the keys but this does not address the issue of key management.
Do you guys have any thoughts on this? or even SaaS/PaaS suggestions?
My recommendation would be to use AWS Secrets Manager for this. Secrets Manager allows you to store any type of credential/key, you can set up fine-grained cross account permissions to secrets, encryption at rest is used (via KMS), and secrets can be automatically rotated (by providing an expiration time and an AWS Lambda function owned by you to perform the rotation).
More details on the official docs:
Basic tutorial on how to use AWS Secrets Manager
Encryption at rest on Secrets Manager
Secrets rotation
Managing secrets policies