I have created an instance with instance profile associated.
I am trying to run packer on the instance without using access and secret key, hence I am using the profile option.
However, the packer requires a profile name, but the instance profile doesn't create a profile name and also doesn't create credentials under ~/.aws.
I can see the access and secret key associated via the AWS CLI, however it doesn't have a profile name.
[ec2-user#ip-172-31-19-132 ~]$ aws configure list
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ****************3IO2 iam-role
secret_key ****************cFvD iam-role
region ap-southeast-2 imds
As I understand from this answer, access and secret keys are rotated frequently, I am wondering how can I set the profile name to the credentials permanently, so I can refer that in packer.
There won't be a profile name as you're using an IAM role. Instead you should natively use the IAM role instead.
Finally, Packer will use credentials provided by the task's or instance's IAM role, if it has one.
This is a preferred approach over any other when running in EC2 as you can avoid hard coding credentials. Instead these are leased on-the-fly by Packer, which reduces the chance of leakage.
Remove the references to the profile option, and it should just natively work just like the SDK / CLI functionality would.
Related
I have an ec2 instance with a specific role,
when typing the awscli command:
[TEST#JenkinsSlave ~]$ aws configure list
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ****************E25I iam-role
secret_key ****************Yl4d iam-role
region <not set> None None
I can see that I have an access key and secret key which I didn't create,
This is a clean amazon Linux ami with no configuration.
I thought it was the role keys but there is no such thing as assigning keys to a role, and every time I create a new server I get different keys so it looks like random keys.
does anyone know how I got those credentials?
and How I can delete them from my configuration (as you can see by Location=None they are not stored in ~/.aws/)
Even if you don't have an instance role assigned, your EC2 instance does have instance-based credentials. You can see these with curl http://169.254.169.254/latest/meta-data/identity-credentials/ec2/security-credentials/ec2-instance.
However, when I run aws configure list, those credentials don't show up. It's possible that you're running an earlier or later version than I am (aws-cli/1.16.153) and the behavior has changed. You can compare the credentials from that curl request to see if they're the same.
You should verify that you don't, in fact, have an instance role set. Either check the console or use curl http://169.254.169.254/latest/meta-data/iam/ from the instance. If you don't have a role set you'll get a 404. If you do have a role set this command will show a file named info, and that file will contain the instance role information.
Edit: the only thing that I've tried that does produce the output you're seeing is to assign an instance profile to the instance.
In response to comment asking about different tokens for each machine: this is how instance profiles work. Each machine gets a time-limited set of credentials that can be used to make requests, and automatically renews those credentials as needed. This is a Good Thing, as it means that those credentials can not be taken from the machine and used to gain unauthorized access to the services.
I have EC2 instance.
I'm trying to call aws s3 from it but getting an error
Unable to locate credentials
I tried aws configure which does show everything as empty.
I see IAM role for S3 full permissions assigned to this instance.
Do I need any additional configuration?
If you run aws on an Amazon EC2 instance that has an assigned role, then it should find the credentials automatically.
You can also use this to view the credentials from the instance:
curl http://169.254.169.254/latest/meta-data/iam/security-credentials/
The role name should be listed. Then append it to the command:
curl http://169.254.169.254/latest/meta-data/iam/security-credentials/ROLE-NAME/
You should then be presented with AccessKeyId, SecretAccessKey, etc.
If that information does not appear, it suggests that your role is not correctly attached to the instance. Try unassigning the role and then assign it to the instance again.
First read and follow John Rotenstein's advice.
The SDK (from which the CLI is built from) searches for credentials in the following order:
Environment variables (AWS_ACCESS_KEY_ID ....)
Default Profile (credentials file)
ECS Container Credentials (if running on ECS)
Instance Profile
After verifying that your EC2 instance has credentials in the metadata, double check 1 & 2 to make sure that there are no other credentials present even if empty.
Note: This link argues with my last point (empty credentials file). I never create (store) credentials on an EC2 instance and I only use IAM Roles.
Instance Metadata
For different AWS services, I need different IAM users to secure the access control. Sometimes, I even need to use different IAM user credentials within a single project in a EC2 instance. What's the proper way to manage this and how I can deploy/attach these IAM user credentials to a single EC2 instance?
While I fully agree with accepted answer that using static credentials is one way of solving this problem, I would like to suggest some improvements over it (and proposed Secrets Manager).
What I would advise as architectural step forward to achieve full isolation of credentials, having them dynamic, and not stored in central place (Secrets Manager proposed above) is dockerizing application and running on AWS Elastic Container Service (ECS). This way you can assign different IAM role to different ECS Tasks.
Benefits over Secrets Manager solution
- use case of someone tampering with credentials in Secrets Manager is fully avoided, as credentials are of dynamic nature (temporary, and automatically assumed through SDKs)
Credentials are managed on AWS side for you
Only ECS Service can assume this IAM role, meaning you can't have actual person stealing the credentials, or developer connecting to production environment from his local machine with this credentials.
AWS Official Documentation for Task Roles
The normal way to provide credentials to applications running on an Amazon EC2 instance is to assign an IAM Role to the instance. Temporary credentials associated with the role when then be provided via Instance Metadata. The AWS SDKs will automatically use these credentials.
However, this only works for one set of credentials. If you wish to use more than one credential, you will need to provide the credentials in a credentials file.
The AWS credentials file can contain multiple profiles, eg:
[default]
aws_access_key_id = AKIAaaaaa
aws_secret_access_key = abcdefg
[user2]
aws_access_key_id = AKIAbbbb
aws_secret_access_key = xyzzzy
As a convenience, this can also be configured via the AWS CLI:
$ aws configure --profile user2
AWS Access Key ID [None]: AKIAbbbb
AWS Secret Access Key [None]: xyzzy
Default region name [None]: us-east-1
Default output format [None]: text
The profile to use can be set via an Environment Variable:
Linux: export AWS_PROFILE="user2"
Windows: set AWS_PROFILE="user2"
Alternatively, when calling AWS services via an SDK, simply specify the Profile to use. Here is an example with Python from Credentials — Boto 3 documentation:
session = boto3.Session(profile_name='user2')
# Any clients created from this session will use credentials
# from the [user2] section of ~/.aws/credentials.
dev_s3_client = session.client('s3')
There is an equivalent capability in the SDKs for other languages, too.
I am mounting an AWS S3 bucket as a filesystem using s3fs-fuse. It requires a file which contains AWS Access Key Id and AWS Secret Access Key.
How do I avoid the access using this file? And instead use AWS IAM roles?
As per Fuse Over Amazon document, you can specify the credentials using 4 methods. If you don't want to use a file, then you can set AWSACCESSKEYID and AWSSECRETACCESSKEY environment variables.
Also, if your goal is to use AWS IAM instance profile, then you need to run your s3fs-fuse from an EC2 instance. In that case, you don't have to set these credential files/environment variables. This is because while creating the instance, if you attach the instance role and policy, the EC2 instance will get the credentials at boot time. Please see the section 'Using Instance Profiles' in page 190 of AWS IAM User Guide
there is an argument -o iam_role=--- which helps you to avoid AccessKey and SecretAccessKey
The Full steps to configure this is given below
https://www.nxtcloud.io/mount-s3-bucket-on-ec2-using-s3fs-and-iam-role/
I want to "associate an instance profile" with my environment (to speed up upload performance for my website) but can't find useful instructions on how anywhere in the AWS documentation (or here on SO).
How do I "associate an instance profile" with an environment on AWS?
Not clear what you are trying to accomplish, but if you are look to just 'associate an instance profile', here you go:
http://docs.aws.amazon.com/IAM/latest/UserGuide/instance-profiles.html
An instance profile is a container for an IAM role. Instance profiles
are used to pass role information to an Amazon EC2 instance when the
instance starts. When you use the Amazon EC2 console to launch an
instance with an IAM role, you can select a role to associate with the
instance. In the console, the list that's displayed is actually a list
of instance profile names.
If you use the AWS Management Console to create a role, the console
creates an instance profile automatically and gives it the same name
as the role it corresponds to. However, if you use the CLI, API, the
AWS SDK, or a third-party tool to create roles and instance profiles,
you create the roles and instance profiles as separate actions, and
you might give them different names. In that case, you need to know
the names of your instance profiles as well as the names of roles they
contain so that you can choose the correct instance profile when you
launch an Amazon EC2 instance.
If you are seeing this error when trying to set up a Elastic Beanstalk environment, it could be that the IAM user does not have sufficient permissions. You should add the IAM Policy named IAMFullAccess for that user.
You can check one option that works here for AWS Beanstalk. This is using AWS Console.
Create IAM Role for "EC2" as AWS Service and attach
"AWSElasticBeanstalkFullAccess" policy to it.
Create an AWS Beanstalk environment with tier/platform/application of choice. (I did with Web Tier choosing Docker platform using python:3-onbuild).
Hit "Configure More Options" > "Modify" Security.
Choose "IAM instance profile" which was created in Step 1. (Keep Service role
"aws-elasticbeanstalk-service-role" as this is needed! It has some key policies!)
"Save" and "Create Environment".
Similarly, it should work for AWS CLI and AWS SDK as well. Key here is the instance profile role you choose for EC2 (or say instance IAM role/instance profile) during Beanstalk Environment creation.
You can create customer managed policy extending or reusing AWSElasticBeanstalkFullAccess policy based on need. Refer AWSElasticBeanstalkFullAccess to understand it better. Also, check out some more AWS Managed policies starting with AWSElasticBeanStalk*.
Somehow default aws-elasticbeanstalk-ec2-role policy is not working/taking effect as desired for me.
For what it's worth, the AWS console now displays an alert (see image) that has a link to help you associate the instance profile with your environment. Therefore, if you see this message in the CLI, you probably need to log into the Management Console, navigate to Elastic Beanstalk, and choose your environment. You will then see the message and be able to use the link to resolve the issue.
Note 1: You may need to redeploy your app after associating the profile. I did.
Note 2: It really does make deployments faster.
Orome,
As E.J Brennan pointed out in his post, you must have a role created first, and attach the necessary IAM policies to that role, then you create the instance profile:
aws iam create-instance-profile --instance-profile-name YOUR_INSTANCE_PROFILE_NAME
Then add the role you created in the beginning to the instance profile:
aws iam add-role-to-instance-profile --instance-profile-name YOUR_INSTANCE_PROFILE_NAME --role-name YOUR_ROLE
Then attach the instance profile to your instance:
aws ec2 associate-iam-instance-profile --instance-id YOUR_INSTANCE_ID --iam-instance-profile Name="YOUR_INSTANCE_PROFILE_NAME"
Is this what you are looking for?
As E.J Brennan also pointed out, the AWS management console does the instance profile for you behind the scenes.