Using AWS Step Functions to string together Lambdas is interesting, but is there a way to disable/hide logging on the execution details screen? Private information being handed from one lambda to another needs to be able to be done in secret, and adding a KMS encrypt/decrypt to each step is a ton of overhead, and impossible for lambdas that live in a VPC without internet access.
We've talked with Amazon and it looks like that there's no way to hide this information from the console. The alternative is to limit what gets sent to the Lambda functions at each step.
So you can ensure that only non-PII subsets of the input data are seen by certain functions. The usual workaround is to not passing PII data in at all and instead of that, place the PII data in an encrypted data stores, such as an S3 bucket or encrypted RDS database table, and pass a reference to that object through the state machine.
Another option is to use SSM parameter store with the SecureString type using KMS encryption. You would pass the name of the SSM parameter between steps. The lambda functions would use the API to retrieve and decrypt the value in a single request. See the link below for documentation on how to work with SSM parameter store using boto3 in Python.
http://boto3.readthedocs.io/en/latest/reference/services/ssm.html#SSM.Client.get_parameter
You will have to ensure that the roles to your lambda functions provide access to the SSM parameters AND access to same KMS key that was used to encrypt the value.
Related
I am trying to hide the "app key id" which define in the Lambda Environment variable using Terraform to show as > ************************* instead of the real value.
Any suggestions how can I don that ?
Thanks
There is no way to achieve what you are trying and lambda environment variables are not a suitable place for storing app secrets.
For storing secrets, I would recommend that you use AWS Secrets Manager or AWS System Manager Parameter Store. Both have the ability to hide/mask the value, and also encrypt it, as per your use case.
AWS Secrets Manager is a costlier service in comparison but provides more features like integration for secret rotation with some AWS services (RDS, RedShift, DocumentDB) but the same effect can be achieved in Parameter Store using lambda functions.
You can compare and contrast and see what fits your use case better!
Encountered a few speicific use cases that I'm somewhat confused to use which:
A large number of free, public API keys. Using lambda environment variable with encyption, other developer/admin can still expose their plaintext value right in the lambds console. Should Parameter Store be used instead?
Login credentials to a third party platform. I assume that Secrets Manager is the only option?
DB Connection strings. Secrets Manager? At $0.40/secret/month, the bill would add up for hundreds of DBs for simply storing credentials.
For storing any credentials you have three AWS managed choices:
Lambda Environment Variables
These will be passed into the Lambda function directly via the Lambda Service. You can prevent others accessing the string values by controlling their permissions to KMS via IAM. This will provide the best performance out of any options (there's no additional lookup in the code runtime).
By using this option be aware of the following pitfalls:
If you use versioning for your Lambda function the values are fixed, you would need to deploy a new version of the Lambda function to make changes.
Values are attached to an individual Lambda function, if the keys are used by multiple you will need to pass to each function individually.
Systems Manager Parameter Store
Using this option you would use the SDK to retrieve any key/values that you want. It can store both plain text values as well encrypted strings (the SecureString type). It provides basic functionality but if that is all you need then it will work great. It costs nothing to store the values, but the price is $0.05 per 10,000 Parameter Store API interactions. Unlike environment variables you can use the value across multiple Lambda functions.
By using this option you would need to be aware of the following:
There will be a hit to performance for retrieving the value everytime, to reduce this call the function in the global context so that it can be reused between invocations.
You will need an individual parameter per each key/value. For a database this would mean either creating individual parameters or storing the entire credential set as JSON object and decoding after you retrieve it.
Secrets Manager
Using this option a lot of the management is built into the service, a secret can contain either a string or a single line JSON object. The SDK will handle the retrieval of these values but you must be aware just like SSM you will take a performance hit so you'll want to take a look at a similar solution as the parameter store. The biggest advantage to secrets manager over SSM parameter store is its integrations with other AWS services allowing features such as secret rotation.
However if you don't need the features of secrets manager you may be paying for more than you actually require, this is the most expensive option of all three.
A large number of free, public API keys. Using lambda environment variable with encyption, other developer/admin can still expose their plaintext value right in the lambds console.
For the issue of developers being able to see the environment variables in the console, you can use a non-default KMS CMK, and configure permissions on that key so that the other developers can't use it (doc). They will still be able to see the rest of the Lambda configuration.
A bigger issue is how you will configure these environment variables. If you're using Terraform, for example, the configuration is written to the state file, and you will need to use external state (stored in S3 or on HashiCorp's servers) to secure it. If you're using CloudFormation, you can configure them using a dynamic reference to a Secrets Manager secret, but not to a Parameter Store secure string.
One other choice is to use the environment variables to reference parameter store keys, and then programmatically retrieve the values. For example, you have an environment variable named DATABASE_PASSWORD, and its value is /dev/database/password; the actual database password is a SecureString in Parameter Store, accessed via that path.
Login credentials to a third party platform. I assume that Secrets Manager is the only option?
Parameter store also provides a SecureString.
DB Connection strings. Secrets Manager? At $0.40/secret/month, the bill would add up for hundreds of DBs for simply storing credentials.
Does your application actually connect to hundreds of DBs? If yes, is $40/month (for 100 connections) really a financial hardship for your company?
If yes, then Parameter Store might be the best choice, but beware that there are a limited number of "free" parameters per account.
I need to access some credentials on lambda functions and one way to do that is to put them in environment variables.
However, the values of these credentials are visible in the lambda console. I am thinking to put them in secret manager or parameter store and put the key as env in lambda. Then load the value in lambda at runtime.
However, this approach is secure but give some latency. I am not sure how much latency it gives. Is there a better solution for that?
The Lambda variables are actually encrypted already, the decrypted values are shown from the console but if the user does not have permission for the key they will not be able to see them.
As you suggested you have the solutions of:
Secrets Manager
Systems Manager Parameter Store (use a SecureString).
The benefits that you will get by using either of these solutions is that you can change them outside of the version you're using as well as across a number of Lambda functions simultaneously.
You will also have an increased latency from trying to reach these service endpoints which would lead to a slight increase in time. You could reduce this latency by having your Lambda within a VPC and use VPC endpoints to the service (which will allow direct private communication over using the public internet) but it will still be longer than environment variables.
Ultimately this choice is for you, if you do need to reuse the variables and can put up with the slight latency then use secrets manager or systems manager parameter store. Otherwise manage the KMS permissions so that not every user can access the get to decrypt.
I have a key that is being shared among different services and it is currently stored in an s3 bucket inside a text file.
My goal is to read that variable and pass it to my lambda service through cloudformation.
for an ec2 instance it was easy because I could download the file and read it, and that was easily achievable by putting the scripts inside my cloudformation json file. But I don't have any idea how to do it for my lambdas....!
I tried to put my credentials in gitlab pipeline but because of the access permissions it doesn't let gitlab pass it on, so my best and least expensive option right now is to do it in cloud formation.
The easiest method would be to have the Lambda function read the information from Amazon S3.
The only way to get CloudFormation to "read" some information from Amazon S3 would be to create a Custom Resource, which involves writing an AWS Lambda function. However, since you already have a Lambda function, it would be easier to simply have that function read the object.
It's worth mentioning that, rather than storing such information in Amazon S3, you could use the AWS Systems Manager Parameter Store, which is a great place to store configuration information. Your various applications can then use Parameter Store to store and retrieve the configuration. CloudFormation can also access the Parameter Store.
My system architecture looks like as follows:-
SNS -> AWS Lambda -> Dynamo Db
So, SNS is publishing messages to which AWS Lambda function is the subscriber and then AWS Lambda pushes the data into Dynamo Db. In this, I am doing some transformation of messages in AWS Lambda. For the transformation, I have to fetch some rules from some place. These rules are basically the mapping between fields of the original messages to fields to transformed messages.
Eg.
Say, Original Message looks like below:-
{"id": 1,
"name":"dsadas",
"house":"dsads dsadsa",
"speciality":"asjdsa"
}
and my mapping is something like:-
{"id":"id",
"house":"home",
"speciality":"area"
}
So, basically I am saying that id should be mapped to id, house to home and so on.
So, I want to keep this mapping at some places like Dynamo Db or some config service. I do not want to directly keep it in aws lambda code as there is a chance that I might have to change. But keeping it in Dynamo Db will be very costly in terms of latency I think because I will make a call on each message request. So, can anyone suggest, any aws resource which can be used for keeping these configs which is very fast and normally used for keeping configuration.
If you need full flexibility of modifying the mapping, without modifying the Lambda code, you have to rely on S3 or Dynamodb or any other storage service to keep the mappings which also adds latency and cost.
You can also keep a separate mapping.json(or js) and upload the file along with your Lambda code. The drawback is, you need to redeploy the Lambda function for each mapping.json modification.
Another option is to use environment variables, only to keep the attribute mapping key value pairs and the mapping template is constructed inside Lambda using these variables.
You can also use base64 encoding of the mapping template to an environment variable and use it in Lambda.