I am writing an AWS Lamda in C# using this example:
https://docs.aws.amazon.com/toolkit-for-visual-studio/latest/user-guide/lambda-creating-project-in-visual-studio.html
I have set my credentials in the environmental variables according to this document:
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html
But when I try to upload my lamda I get errors about not having the correct IAM roles. However, the account name in the error message is not the same as the account I put into the credentials in the environmental variables. It relates to an account on a different S3 bucket which I was testing with little while ago.
Where is it reading this from?
I have set my credentials in the environmental variables according to this document
This is not a good practice. The lambda permissions should be provided using lambda execution role. Once you set the role, AWS SDK will automatically get the temporary credentials from your function's runtime environment variables.
Since it seems that you need a cross-account access in your function, then you have to use cross-account roles as explained in AWS docs:
How do I configure a Lambda function to assume an IAM role in another AWS account?
Related
I am trying to follow best practices, but the documentation is not clear to me. I have a python script running locally that will move some files from my local drive to S3 for processing. Lambda picks it up from there and does the rest. So far I set up an AWS User for this process, and connected it to a "policy" that only has access to the needed resources.
Next step is to move my scripts to a docker container in my local server. But I thought best practice would be to use a Role with policies, instead of a User with policies. However, according to this documentation... in order to AssumeRole... I have to first be signed in as a user.
The calls to AWS STS AssumeRole must be signed with the access key ID
and secret access key of an existing IAM user or by using existing temporary
credentials such as those from another role. (You cannot call AssumeRole
with the access key for the root account.) The credentials can be in
environment variables or in a configuration file and will be discovered
automatically by the boto3.client() function.
So no matter what, I'll need to embed my user credentials into my docker image (or at least a separate secrets file)
If that is the case, then it seems adding a "Role" in the middle between the User and the Policies seems completely useless and redundant. Can anyone confirm or correct?
Roles and policies are for services running in AWS environments. For a Role you define a Trust Policy. The Trust Policy defines what principal (User, Role, AWS Service etc.) can assume it. You also define the permissions that the principal which assumes it has to access AWS services.
For services running inside AWS (EC2, Lambda, ECS), it is always possible to select an IAM role, which will be assumed by your service. This way your application will always get temporary credentials corresponding to the IAM role and you should never use an AWS Access Key Id and Secret.
However, this isn't possible for services running locally or outside of AWS environment. For your Docker container running locally, the only real option would be to create an Access Key ID and Secret and copy it there. There are still some things you can do to keep your account secure:
Follow the least privilege principal. Create a policy that provides access to only the absolutely required resources.
Create a user (programmatic access only) and add the policy. Use AWS Access Key ID and Secret of this user for your Docker container.
Make sure that the AWS Credentials are rotated regularly.
Make sure that the secrets aren't committed in source control, prefer a secrets file or a Vault system than environmental variables.
I am trying to connect to an AWS AppSync, located in a custom profile/account/role under the root/default account, from an AWS Lambda in the root/default account.
The below python code works fine locally, because I have configured "custom_profile" in my local .aws/config file.
session = boto3.Session(profile_name='custom_profile')
client = session.client('appsync', region_name='<region>')
But is there any way to make this code run in the AWS Lambda in the root account? How can AWS Lambda understand what is "custom_profile"? Where and how can I map "custom_profile" to the respective role ARN?
I saw a probable solution to this problem on this link, but I have not tried it.
Has anyone faced a similar issue and know of an easier solution to this problem than in the link?
The link that you've referenced is the way to go. Permissions that an AWS Lambda function has, are to be defined in a role for that function. This can include permissions to assume a role in another account.
You can then use the Security Token Service (or STS for short) and execute the AssumeRole action. This will provide you with AWS tokens that you can use to authenticate your calls to the other account.
You will also have to configure the account you're executing the lambda function in as a trusted entity in the role you want to assume in the second account.
I have a cross account IAM question about running in a Lambda function. (I know people may use STS assume, but this package really isn't worth protecting, and I don't want to go through linking the accounts)
Account “A”.
S3 – package “s3://foo/foo”
IAM credentials “pkg_creds” for bucket "s3://foo"
Account “B”
Lamba function “gogo” runs
In this Lambda function, it attempts to use boto3 and the pkg_creds to
download package “s3://foo/foo”, but if fails with this error:
**The provided token is malformed or otherwise invalid.**
Lambda is read only, but I believe boto3 will not write credentials to ~/.aws if I'm using boto3.client (not session). However, I also set the AWS_CONFIG_FILE to /tmp just in case. It still fails. I suspect what I'm proposing isn't possible because LAMBDA has immutable AWS credentials, where you can't change scopes, even one that is explicitly given to boto3.
Let me know your thoughts. I may try do the job with Faragate, but Lambda function is easier to maintain and deploy.
Thanks in advance!
Lambda isn't using a ~/.aws config file at all, it is using environment variables by default. There are many ways to configure AWS credentials in boto3. You should be able to create a new boto3 client in your Lambda function with explicit AWS credentials like so:
client = boto3.client(
's3',
aws_access_key_id=ACCOUNT_A_ACCESS_KEY,
aws_secret_access_key=ACCOUNT_A_SECRET_KEY
)
And pass ACCOUNT_A_ACCESS_KEY and ACCOUNT_A_SECRET_KEY as environment variables to the function.
User error. I can verify boto3 in a lambda function can use credentials outside of its scope.
after troubleshooting more. the issue was that i was took in the "environmental variable" SESSION which is set on the lambda function, but not on my ec2 instance. so i was always using the lambda session key that seems to overide the explicit key and secret.
I am deploying a server program in an ec2 instance which needs to be able to create pre-signed urls for s3. So far I've had my AWS credentials in environment variables for testing, but I would like to switch to the IAM Role strategy now. However, I am unsure as to which policies the role should have access too. My initial guess is to have AmazonS3FullAccess, but the description says "Provides full access to all buckets via the AWS Management Console" but the ec2 instance will be using the c++ sdk, not the management console. Or is the policy not important, just that it has a policy so it gets credentials somehow?
You're confusing policies and roles.
a policy grants permissions to a user or to a role or to a group.
the difference between a user and a role is subtle, but basically a role is something that's assumed by other services in AWS, like an EC2 instance, while a user is generally just an identity you've created for use in AWS.
The policy description for full access may make mention to the management console, but it grants full access to all buckets whether through the console, the api or an sdk, they're all really the same thing under the hood.
You should not use the fullaccess policy. You could use it as a base to build your real policy, but IAM should always use the least privilege principal, where you only give the permissions that are absolutely required, in this case the role only needs read and possibly list permissions on the specific bucket in question if generating urls for reading, or put permissions if allowing uploads.
According to many advices, we should not configure IAM USER but using IAM Role instead to avoid someone managed to grab the user confidential in .aws folder.
Lets say I don't have any EC2 instances. Can I still able to perform S3 operation via AWS CLI? Says aws s3 ls
MacBook-Air:~ user$ aws s3 ls
Unable to locate credentials. You can configure credentials by running "aws configure".
You are correct that, when running applications on Amazon EC2 instances or as AWS Lambda functions, an IAM role should be assigned that will provide credentials via the EC2 metadata service.
If you are not running on EC2/Lambda, then the normal practice is to use IAM User credentials that have been created specifically for your application, with least possible privilege assigned.
You should never store the IAM User credentials in an application -- there have been many cases of people accidentally saving such files into GitHub, and bad actors grab the credentials and have access to your account.
You could store the credentials in a configuration file (eg via aws configure) and keep that file outside your codebase. However, there are still risks associated with storing the credentials in a file.
A safer option is to provide the credentials via environment variables, since they can be defined through a login profile and will never be included in the application code.
I don't think you can use service roles on your personal machine.
You can however use multi-factor authentication for AWS CLI
You can use credentials on any machine not just EC2.
Follow the steps as described by the documentation for your OS.
http://docs.aws.amazon.com/cli/latest/userguide/installing.html