I am using latest version of s3cmd 1.5.2. As per this post, 1.5.0-alpha2 supports IAM Roles, so I am assuming this must be supported in 1.5.2 as well. I added "security_token" in s3cfg file as mentioned in this post. But I am still getting error "ERROR: S3 error: The AWS Access Key Id you provided does not exist in our records." Am I doing something wrong or this version doesn't support IAM Roles?
I figured out the answer myself. After diving into the s3cmd code, found out that instead of adding "security_token" as mentioned in the post and other sources, we need to add "access_token" in the s3cfg file. If you are using s3cmd on a cluster associated with a role then there is no need to pass any s3cfg file as well. All releases from 1.5.0-alpha2 and above support IAM Roles.
These days, it is recommended to use the AWS Command-Line Interface (CLI).
As with any software that uses the AWS SDK, the AWS CLI will automatically use role-based credentials if the Amazon EC2 instance is launched with a role. Credentials can also be specified via environment variables or a configuration file.
Related
When using a simple AWS credentials file like this on Ubuntu Linux with credentials file: ~/.aws/credentials
[leigh.m] # let's call this account ID: 1000 for simplicity
AWS_ACCESS_KEY_ID=some_access_key_id
AWS_SECRET_ACCESS_KEY=some_secret_access_key
AWS Toolkit in VSCode works perfectly and I can access AWS resources via VSCode.
However I need to assume roles cross-accounts (with a source 'root' profile controlling access to the other accounts via roles) which all have MFA set.
My ~/.aws/config file that facilitates this looks like:
[profile my-assumed-role-1] # let's say this has account ID: 2000
role_arn=arn:aws:iam::2000:role/AdminRole
source_profile=leigh.m
role_session_name=my-assumed-role-1-session-name
mfa_serial=arn:aws:iam::1000:mfa/leigh.m
duration_seconds=3600
This setup works perfectly well now and I can assume the role for example via CLI I can list buckets in account 2000 by passing that profile, like: aws s3api list-buckets --profile my-assumed-role-1
However, when I try to access that named profile via AWS Toolkit in VSCode I get the error message in AWS toolkit logs:
Shared Credentials Profile leigh.m is not valid. It will not be used by the toolkit.
Failed to connect with "profile:my-assumed-role-1": Profile leigh.m is not a valid Credential Profile: Profile leigh.m is not supported by the Toolkit.
It appears to be looking for a Profile 'leigh.m', so I have tried adding to ~/.aws/config this section:
[profile leigh.m]
mfa_serial=arn:aws:iam::1000:mfa/leigh.m
region=eu-west-1
output=json
I also tried to not use the config file and place everything in credentials file (as it appears to suggest here: https://docs.aws.amazon.com/toolkit-for-visual-studio/latest/user-guide/mfa-credentials.html)
But still the same issue. Anything I have missed?
(I did have a look to see if there were any questions already on S/O on correct setup of AWS credentials for AWS Toolkit and MFA (as my AWS credentials and config files work correctly for all other requirements), this was the only question from ~5 years ago: Visual Studio AWS toolkit with multifactor authentication? but did not answer and no questions/ answers on AWS Toolkit, MFA and assuming roles with names profiles)
This is a VSCode AWS Toolkit bug who are releasing a fix for this: https://github.com/aws/aws-toolkit-vscode/issues/2394
Summary: AWS Toolkit only works with lowercase keys, so this will not work:
AWS_ACCESS_KEY_ID=some_access_key_id
AWS_SECRET_ACCESS_KEY=some_secret_access_key
When the fix is released it will work with caps & lowercase, of course in the short term just change credentials file to:
aws_access_key_id=some_access_key_id
aws_secret_access_key=some_secret_access_key
It will work as expected
I'm learning AWS using the AWS CLI and LocalStack.
I'm working with tutorials such as this, which describe how to create a S3 bucket and upload a file to it.
What I'd like to understand is the role of users in relation to AWS commands, and whether or not there is a relationship between a user and a profile (the latter is created when you run the aws configure CLI command).
When I run a AWS CLI command such as aws --endpoint-url=http://localhost:4572 s3 cp ./foo.json s3://my-bucket/path/to/foo.json what user am I running that command as? I have not explicitly created any users using the AWS IAM CLI or by other means. Is a profile implicitly a user? I.e. when I run aws configure, does the default profile created mean a user named default is created in AWS IAM?
Well, it's easy to check actually.
https://docs.aws.amazon.com/cli/latest/reference/sts/get-caller-identity.html
Is a profile implicitly a user?
No, profile is just that - credentials profile. Whether it's a user or a role, AWS CLI doesn't care as long as those credentials are valid.
when I run aws configure, does the default profile created mean a user named default is created in AWS IAM
Going from previous point, no. Nothing is done implicitly in IAM. When you run aws configure you supply credentials that already exist, not the other way around.
I have multiple roles in AWS and have to switch between them multiple times a day.
Can you please suggest a good solution without doing the entire authentication cycle?
You can store multiple roles in the ~/.aws/credentials file. These can be activated via the AWS CLI by using --profile.
You can also put roles in specific profiles and the AWS CLI will assume them from a base set of credentials.
See:
Configuration and Credential File Settings - AWS Command Line Interface
Using an IAM Role in the AWS CLI - AWS Command Line Interface
I have an AWS root account and I created a IAM user with Administrator policy attached. The problem is that when I make some operations like aws ec2 create-key-pair in the aws-cli, I got UnauthorizedOperation error. But if I sign into the management console using the same IAM user, I can create a key pair with no problem.
I double checked my access key id and secret access key is right. I even re-created these and configured in aws-cli to make sure I am using that user account with admin permissions. None of these worked
Is there any setting I'm not aware of that enables aws-cli access permissions?
=============== below is latest update ================
I checked my user info with aws sts get-caller-identify, and it shows my company's account id. I tried to run aws configure again and followed by aws sts get-caller-identity, it still shows that im using my company's account. How do I switch user from CLI?
In case this helps anyone, the issue for me was that my AWS account used MFA. I never considered you'd have to do this on the command-line too. Here is a guide from Amazon on the subject:
How do I use an MFA token to authenticate access to my AWS resources through the AWS CLI?
This is an incredibly big pot hole to step into... but there is a precedence of where aws cli looks for credentials.
IAM credentials not found on AWS CLI
The answer of the question here explains it throughly.
I'm very new to jenkins. I installed the S3 plugin. I wish to use AWS IAM S3 roles to make the plugin work.
However, I can't find any way to configure IAM roles within this plugin. I can see only setting up the access key and secret key.
Has anyone used IAM roles with this plugin?
Kindly help.
Thanks in advance.
IAM roles are assigned to EC2 instances at launch. There's nothing else to configure, thus your S3 plugin should just automatically work. This is dependent on the plugin supporting IAM roles, which it looks like it does according to your link. You will have to assign the correct bucket policy though using the instructions also found at the link.