I have a cross account IAM question about running in a Lambda function. (I know people may use STS assume, but this package really isn't worth protecting, and I don't want to go through linking the accounts)
Account “A”.
S3 – package “s3://foo/foo”
IAM credentials “pkg_creds” for bucket "s3://foo"
Account “B”
Lamba function “gogo” runs
In this Lambda function, it attempts to use boto3 and the pkg_creds to
download package “s3://foo/foo”, but if fails with this error:
**The provided token is malformed or otherwise invalid.**
Lambda is read only, but I believe boto3 will not write credentials to ~/.aws if I'm using boto3.client (not session). However, I also set the AWS_CONFIG_FILE to /tmp just in case. It still fails. I suspect what I'm proposing isn't possible because LAMBDA has immutable AWS credentials, where you can't change scopes, even one that is explicitly given to boto3.
Let me know your thoughts. I may try do the job with Faragate, but Lambda function is easier to maintain and deploy.
Thanks in advance!
Lambda isn't using a ~/.aws config file at all, it is using environment variables by default. There are many ways to configure AWS credentials in boto3. You should be able to create a new boto3 client in your Lambda function with explicit AWS credentials like so:
client = boto3.client(
's3',
aws_access_key_id=ACCOUNT_A_ACCESS_KEY,
aws_secret_access_key=ACCOUNT_A_SECRET_KEY
)
And pass ACCOUNT_A_ACCESS_KEY and ACCOUNT_A_SECRET_KEY as environment variables to the function.
User error. I can verify boto3 in a lambda function can use credentials outside of its scope.
after troubleshooting more. the issue was that i was took in the "environmental variable" SESSION which is set on the lambda function, but not on my ec2 instance. so i was always using the lambda session key that seems to overide the explicit key and secret.
Related
I am writing an AWS Lamda in C# using this example:
https://docs.aws.amazon.com/toolkit-for-visual-studio/latest/user-guide/lambda-creating-project-in-visual-studio.html
I have set my credentials in the environmental variables according to this document:
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html
But when I try to upload my lamda I get errors about not having the correct IAM roles. However, the account name in the error message is not the same as the account I put into the credentials in the environmental variables. It relates to an account on a different S3 bucket which I was testing with little while ago.
Where is it reading this from?
I have set my credentials in the environmental variables according to this document
This is not a good practice. The lambda permissions should be provided using lambda execution role. Once you set the role, AWS SDK will automatically get the temporary credentials from your function's runtime environment variables.
Since it seems that you need a cross-account access in your function, then you have to use cross-account roles as explained in AWS docs:
How do I configure a Lambda function to assume an IAM role in another AWS account?
I am trying to connect to an AWS AppSync, located in a custom profile/account/role under the root/default account, from an AWS Lambda in the root/default account.
The below python code works fine locally, because I have configured "custom_profile" in my local .aws/config file.
session = boto3.Session(profile_name='custom_profile')
client = session.client('appsync', region_name='<region>')
But is there any way to make this code run in the AWS Lambda in the root account? How can AWS Lambda understand what is "custom_profile"? Where and how can I map "custom_profile" to the respective role ARN?
I saw a probable solution to this problem on this link, but I have not tried it.
Has anyone faced a similar issue and know of an easier solution to this problem than in the link?
The link that you've referenced is the way to go. Permissions that an AWS Lambda function has, are to be defined in a role for that function. This can include permissions to assume a role in another account.
You can then use the Security Token Service (or STS for short) and execute the AssumeRole action. This will provide you with AWS tokens that you can use to authenticate your calls to the other account.
You will also have to configure the account you're executing the lambda function in as a trusted entity in the role you want to assume in the second account.
I'm writing a lambda function in Python 3.8. The function connects with a dynamodb using boto3:
db = boto3.resource('dynamodb', region_name='foo', aws_access_key_id='foo', aws_secret_access_key='foo')
That is what I have while I am developing on my local machine and need to test the function. But, when I deploy this to lambda, I can just remove the credentials and my function will connect to the dynamodb if I have the proper IAM roles and policies setup in place. For example, this code would work fine when deployed to lambda:
db = boto3.resource('dynamodb', region_name='foo')
The question is, how can I manage this in terms of pushing code to lambda? I am using AWS SAM to deploy to AWS. Right now what I do is once I'm done developing my function, I remove the aws_access_key_id='foo' and aws_secret_access_key='foo' parts manually and then deploy the functions using SAM.
There must be a better way to do this? Could I embed these into my IDE instead? I'm using PyCharm. Would that be a better way? If not, what else?
You should never put credentials in the code like that.
When running the code locally, use the AWS CLI aws configure command to store local credentials in a ~/.aws/config file. The AWS SDKs will automatically look in that file to obtain credentials.
In sam you can invoke your function locally using sam local invoke or sam local start-lambda.
Both of them take --profile parameter:
The AWS credentials profile to use.
This will ensure that your local lambda environment executes with correct credentials without needing to hard code them in your code. Subsequently, you can test your code without modifications which would otherwise be needed when hard coding the key id and secret key.
You can use environment variables.
Environment variables can be configured both in pycharm, as well as in AWS Lambda and AWS SAM.
As stated in the Lambda best practices: "Use environment variables to pass operational parameters to your function. For example, if you are writing to an Amazon S3 bucket, instead of hard-coding the bucket name you are writing to, configure the bucket name as an environment variable."
You can also use an environment variable to specify which environment is being used, which can then be used to explicitly determine whether credentials are necessary.
My Aws Lambda function is written in Java. I am getting data from DynamoDb by giving some static credentials like below;
new BasicAWSCredentials(ACCESSKEY, SECRETKEY)
However, when i try to define my services in Aws Cloudformation. I could not find any way, how can i change these accesskey and secretkey credentials. What is the best way for managing these credentials?, Because they are special keys for each account and embedded in Java code.
Although you could use the context or download a file to pass credentials in runtime, one should not use explicit hard-coded credentials, as that is harder to acquire and rotate securely.
It is easier and safer to use roles, as described in the lambda permission model: http://docs.aws.amazon.com/lambda/latest/dg/intro-permission-model.html
Use explicit credentials only outside AWS (your dev machine for example), and even so do not hard-code them, use environment variables or CLI profiles.
I have a lambda function configured through the API Gateway that is supposed to hit an external API via Node (ex: Twilio). I don't want to store the credentials for the functions right in the lambda function though. Is there a better place to set them?
The functionality to do this was probably added to Lambda after this question was posted.
AWS documentation recommends using the environment variables to store sensitive information. They are encrypted (by default) using the AWS determined key (aws/lambda) when you create a Lambda function using the AWS Lambda console.
It leverages AWS KMS and allows you to either: use the key determined by AWS, or to select your own KMS key (by selecting Enable encryption helpers); you need to have created the key in advance.
From AWS DOC 1...
"When you create or update Lambda functions that use environment variables, AWS Lambda encrypts them using the AWS Key Management Service. When your Lambda function is invoked, those values are decrypted and made available to the Lambda code.
The first time you create or update Lambda functions that use environment variables in a region, a default service key is created for you automatically within AWS KMS. This key is used to encrypt environment variables. However, should you wish to use encryption helpers and use KMS to encrypt environment variables after your Lambda function is created, then you must create your own AWS KMS key and choose it instead of the default key. The default key will give errors when chosen."
The default key certainly does 'give errors when chosen' - which makes me wonder why they put it into the dropdown at all.
Sources:
AWS Doc 1: Introduction: Building Lambda Functions » Environment Variables
AWS Doc 2: Create a Lambda Function Using Environment Variables To Store Sensitive Information
While I haven't done it myself yet, you should be able to leverage AWS KMS to encrypt/decrypt API keys from within the function, granting the Lambda role access to the KMS keys.
Any storage service or database service on AWS will be able to solve your problem here. The question is what are you already using in your current AWS Lambda function? Based on that, and the following considerations:
If you need it fast and cost is not an issue, use Amazon DynamoDB
If you need it fast and mind the cost, use Amazon ElastiCache (Redis or Memcache)
If you are already using some relational database, use Amazon RDS
If you are not using anything and don't need it fast, use Amazon S3
In any case, you need to create some security policy (either IAM role or S3 bucket policy) to allow exclusive access between Lambda and your choice of storage / database.
Note: Amazon VPC support for AWS Lambda is around the corner, therefore any solution you choose, make sure it's in the same VPC with your Lambda function (learn more at https://connect.awswebcasts.com/vpclambdafeb2016/event/event_info.html)
I assume you're not referring to AWS credentials, but rather the external API credentials?
I don't know that it's a great place, but I have found posts on the AWS forums where people are putting credentials on S3.
It's not your specific use-case, but check out this forum thread.
https://forums.aws.amazon.com/thread.jspa?messageID=686261
If you put the credentials on S3, just make sure that you secure it properly. Consider making it available only to a specific IAM role that is only assigned to that Lambda function.
For 2022 we have AWS Secrets Manager for storing sensitive data like Database Credentials, API Tokens, Auth keys, etc.