By googling, I found this tutorial on accessing S3 from EC2 instance without credential file. I followed its instructions and got the desired instance. The aws web console page looks like
However, I don't want to do it manually using the web console every time. How can I create such EC2 instances using boto3?
I tried
s = boto3.Session(profile_name='dev', region_name='us-east-1')
ec2 = s.resource('ec2')
rc = ec2.create_instances(ImageId='ami-0e297018',
InstanceType='t2.nano',
MinCount=1,
MaxCount=1,
KeyName='my-key',
IamInstanceProfile={'Name': 'harness-worker'},
)
where harness-worker is the IAM role with access to S3, but nothing else.
It is also used in the first approach with the aws web console tutorial.
Then I got error saying
ClientError: An error occurred (UnauthorizedOperation) when calling
the RunInstances operation: You are not authorized to perform this
operation.
Did I do something obviously wrong?
The dev profile has AmazonEC2FullAccess. Without the line IamInstanceProfile={'Name': 'harness-worker'},, create_instances is able to create instance.
To assign an IAMProfile to an instance, AmazonEC2FullAccess is not sufficient. In addition, you need the following privilege to pass the role to the instance.
See: Granting an IAM User Permission to Pass an IAM Role to an Instance
{
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "*"
}
First you can give full IAM access to your dev profile and see it works. Then remove full IAM access and give only iam:PassRole and try again.
This has nothing to do with the role you are trying to assign the new EC2 instance. The Python script you are running doesn't have the RunInstances permission.
Related
I ran into a problem with AWS instance when I was trying to import self signed SSL certificate to IAM console following this tutorial -> https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/configuring-https-ssl.html
Basically tutorial is made to self sign a certificate and upload it to IAM user to have HTTPS application for testing purposes.
I SSH to my instance and ran all those commands, but in the end when I need to import it I get the error that my account is not authorized...
An error occurred (AccessDenied) when calling the
UploadServerCertificate operation: User:
arn:aws:sts::xxxxxxxxx:assumed-role/aws-elasticbeanstalk-ec2-role/xxxxxxx
is not authorized to perform: iam:UploadServerCertificate on resource:
arn:aws:iam::xxxxxxxxx:server-certificate/elastic-beanstalk-x509
I'm logged in as a ec2-user into the instance because I didn't find a way to log in with any other user...
I tried running command as sudo and nothing changes. On a similar post I have seen that I need to create a specific IAM user to which I need to append specific group policy to have "IAMFullAccess" policy. But I don't understand how can I specify that I want to run this command as this user since I am logged in as ec2-user on SSH...
You need to do some reading: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html
Create an IAM role with Upload permission
Add a trust policy to the role that it will allow it to be assumed by your EC2 instance
Attach the role to the EC2 instance
From your error it seems that you are using Elastic Beanstalk. This means that you already have a role that is assumed by your EC2. Find this role (xxxxx in the error message) and add the appropriate permissions.
Okay I have managed to add the certificate to the instance...
aws iam list-server-certificates {
"ServerCertificateMetadataList": [
{
"ServerCertificateId": "id",
"ServerCertificateName": "elastic-beanstalk-x509",
"Expiration": "2022-10-21T13:07:11Z",
"Path": "/",
"Arn": "arn",
"UploadDate": "2021-10-21T13:42:39Z"
}
] }
I also added Listener and proces on "Modify Application Load Balancer" but the site is still not responding to HTTPS requests... Any idea?
When I run aws command like aws s3 ls, it uses default profile. Can I create a new profile to use a role attached to EC2 instance?
If so, how can I write credentials/config files?
From Credentials — Boto 3 Docs documentation:
The mechanism in which boto3 looks for credentials is to search through a list of possible locations and stop as soon as it finds credentials. The order in which Boto3 searches for credentials is:
Passing credentials as parameters in the boto.client() method
Passing credentials as parameters when creating a Session object
Environment variables
Shared credential file (~/.aws/credentials)
AWS config file (~/.aws/config)
Assume Role provider
Boto2 config file (/etc/boto.cfg and ~/.boto)
Instance metadata service on an Amazon EC2 instance that has an IAM role configured.
Since the Shared Credential File is consulted before the Instance Metadata service, it is not possible to use an assigned IAM Role if a credentials file is provided.
One idea to try: You could create another user on the EC2 instance that does not have a credentials file in their ~/.aws/ directory. In this case, later methods will be used. I haven't tried it, but using sudo su might be sufficient to change to this other user and use the IAM Role.
Unfortunately if you have a credentials file, use the environment variables or specify the IAM key/IAM secret via the SDK these will always take a higher precedence than the using the role itself.
If the credentials are required infrequently you could create another role that the EC2s IAM role can assume (using sts:AssumeRole) whenever it needs to perform these interactions. You would then remove the credentials file on disk.
If you must have a credentials file on the disk, the suggestion would be to create another user on the server exclusively for using these credentials. As a credentials file is only used by default for that user all other users will not use this file (unless explicitly stated within the SDK/CLI interaction as an argument).
Ensure that your local user that you create is locked down as much a possible to reduce the chance of unauthorized users gaining access to the user and its credentials.
This is what we solved this problem.I write this answer in case this is valuable for other people looking for answer.
Add a role "some-role" to a instance with id "i-xxxxxx"
$ aws iam create-instance-profile --instance-profile-name some-profile-name
$ aws iam add-role-to-instance-profile --instance-profile-name some-profile-name --role-name some_role
$ aws ec2 associate-iam-instance-profile --iam-instance-profile Name=some-profile-name --instance-id i-xxxxxx
Attach "sts:AssumeRole"
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"ec2.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
}
$ aws iam update-assume-role-policy --role-name some-role --policy-document file://policy.json
Define profile in the instance
Add "some-operator-profile" to use EC2 instace role.
~/.aws/config
[profile some-operator-profile]
credential_source = Ec2InstanceMetadata
Do what you want with the EC2 provided role
$ aws --profile some-operator-profile s3 ls
Following is a quick summary of the question. Read the full description section for the underlying details.
Condensed description:
Assume you have an IAM user already existing and the user is already able to access other AWS services, such as S3, CloudFront, ECS, EC2...
Let's say we need to provide the user with read-only access over the RDS cluster and set up IAM DB Authentication as well.
We perform all the steps mentioned as per the official guide, in OUR local system and it works perfectly and we are able to generate correct auth token for db_user.
However, here is where it gets interesting.. when the user tries to generate the token for the db_user account, from their local machine.. the user will be denied access.
Full description:
Setup:
My RDS cluster instance runs the Aurora MySQL engine.
Engine version: 5.6.10a
I've been following the AWS knowledge center guide on How do I allow users to connect to Amazon RDS with IAM credentials?
The guide doesn't explicitly mention but while generating the authentication token, AWS CLI uses IAM credentials stored locally, to sign the request.
I'd like to highlight that in the below-mentioned snippet, admin is the profile name stored by AWS CLI for my admin IAM user while the db_user is the IAM user (with rds-db:connect privileges).
TOKEN="$(aws --profile admin rds generate-db-auth-token -h.. .. .. -u db_user)
Using the above snippet I'm able to authenticate with the generated token and connect to the cluster.
If --profile attribute is not mentioned, it reads the default profile saved in the credentials file.
Issue:
Instead of using --profile admin I'm looking to use an already existing non-admin IAM profile for generating an authentication token.
For instance, assume IAM user named developer, with RDS read-only privileges and the credentials stored locally under the profile rds_read_only
TOKEN="$(aws --profile rds_read_only rds generate-db-auth-token -h.. .. .. -u db_user)
If I use the above token, I get the following error:
ERROR 1045 (28000): Access denied for user 'db_user'#'ip' (using password: YES)
After hours of troubleshooting, I was able to conclude that my rds_read_only profile is unable to generate valid authentication tokens probably because IAM user developer is missing some required policies.
I tried attaching all policies available under RDS and RDS Data API (individually as well as in combinations) to IAM user developer, without any luck. If I attach the AdministrativeAccess policy to IAM user developer, only then it is able to generate the token successfully.
Question:
What are the mandatory policies required for non-admin IAM users to generate an authentication token successfully?
i saw your question in AWS Blog.
You need to create an IAM policy to define access to your AWS RDS instances. Check this docs
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"rds-db:connect"
],
"Resource": [
"arn:aws:rds-db:us-east-2:1234567890:dbuser:/"
]
}
]
}
Create User inside RDS DB instance with the instructions to use an IAM Plugin. Check this docs
Create the token check this docs
I found this nice plugin that builds a JDBC jar to allow IAM authentication.
This answers the specific question of #Ronnie regarding the Token generation.
Ronnie i am back. I used the following policy in my AWS Account: Sandbox I am an AWS Federated user with AssumeRole priveleges so Admin
You have to be very careful because as you said the article doesn't make the distinction from:
AWS IAM USER using a generated token to access the DB and
AWS IAM User/Role with the right Admin policies that generates a VALID Token
I will give an Example of how to identify the correct generated Tokens. For some reason AWS generates a value but it doesn't tell you whether is a useful token or not :-\
Token without admin special access= WILL NOT WORK
sandboxdb.asdasdffw.ap-southeast-2.rds.amazonaws.com:3306/?Action=connect&DBUser=human_user&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA5XZIHS3GYVMRPRZF%2F20200424%2Fap-southeast-2%2Frds-db%2Faws4_request&X-Amz-Date=20200424T035250Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=3efd467d548ea05a8bdf097c132b03661680908f723861e45323723c870ef646
Token with Access= Will Work! Look carefully and it contains X-Amz-Security-Token=
sandboxdb.ras21th1z8.ap-southeast-2.rds.amazonaws.com:3306/?Action=connect&DBUser=human_user&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIA5XZIHS3GSAQI6XHO%2F20200424%2Fap-southeast-2%2Frds-db%2Faws4_request&X-Amz-Date=20200424T040756Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEEQaDmFwLXNvdXRoZWFzdC0yIkgwRgIhAOzVIondlMxYkJG5nWNeQlxS0M6B1pphgD1ewFwx2VfKAiEAkcp2jNHHmNMgwqUholnW545MwjzoEjS1uh4BHI4R4GAqvgMIbRABGgw5NDQ0NDAzMTc2NDUiDEvFkyEy833kd%2By4nyqbAybqK5dcP0nTlqZ19I2OVZxzwzz%2BUv9RVdVLMPHE5b%2FqXQGVG1CRtw90r9Lt4QkzTBeIVzdtIkXbpwFtqFh24Djb%2BiZHfvElj%2Fhz29ExzStU0fPYMewEB1u%2F2Osi72Fw6KbZ6TDy5EjuWcrrS08PZQ9CHc%2Fc8iDAIKs28vJ70KKcmow0SInVZGHGpD2JAgIL7jvnadVlcAW7lN2OAnxS72Kb4neqNuHcWzfPLfbXaOP1OaOs7vCR7zDlTTxX2aHoVflC69K9K67BqzdnDnnju%2F4XWQWU3r%2ByXylExwOsiG3y4Qq6wv002l%2BpQmF5%2BMXdTrFR5ewpfrcHf8TZLI5eq8HLA2gG1%2B255L%2Bqt%2BD80T%2FCzEdKSJPjppdYSq9FdeCMRSsqp5PpXP%2BDbQZwmhxiE2RmrbOKwNsFPJqUUnemQHXYLB8lily56nnswT2PYmQOGHqnZWRrv%2FTlGOAGlThuiR%2BLhQLBC08nBEGbBqK%2FjU4JwFMY4JfhgUHr8BA9CuGwAu0qIAFzG71M3HzCNX6o56k1gYJB%2F3%2FJaKlp7TCIxIn1BTrqASqywcfKrWhIaNX3t%2BV%2FZoYYO%2FtGVBZLyr3sSmByA%2Fwq538LiPHA0wDE3utOg%2FwNP%2BQGTcXhk1F%2BI0HOHztAQ2afnKW8r1oRbXxYAzb2j2b8MNEwrsaBju2gHFRgZHkM8YI%2FP5cvYr%2F8FQXWcE9eqjdme0hOo3rPETzxZfRwNQTHEntBbVVD1ec0d7DblfSEDZhLk%2By1%2BFMAYf7NeBIfU6GNsAN2hTdSkPPuto2fQKzRybRAwxQz5P3cO5CClUNIxu4J3bM1MUUTux%2BtMjqRvjGxDhB4yLIJmIPOOYLDSOXl3aWO2y4v89wu5A%3D%3D&X-Amz-Signature=1c6fcc472bb2af09055117075ca21d4a5f715910443115116c9230905721e79d
AWS IAM policy For DB User to Connect to AWS RDS DB Instance
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "rds-db:connect",
"Resource": "*"
}
]
}
To avoid messing with local configs i used the following testing process:
Token generation from an AWS EC2 instance running in the same VPC as the DB. Generation is successful
Token generation from local machine by using a Docker container with AWS CLI Token generation is successful.
Of course my user was created in the MySQL DB with the following command
mysql -h $HOSTNAME -u$ADMIN_USER -p$ADMIN_PASS <<EOF
CREATE USER db_human_user IDENTIFIED WITH AWSAuthenticationPlugin AS 'RDS';
GRANT SELECT ON $SPECIFIC_GIVEN_DB.* TO 'db_human_user';
EOF
I want to create tags from within the running ec2 instance, for that I need credentials and I wanted to use the credentials available at curl -s http://169.254.169.254/latest/meta-data/identity-credentials/ec2/security-credentials/ec2-instance. I set access key, secret key and session token as env variables from the above url . Now I tried
aws ec2 create-tags --resources i-instanceid --tags Key=Test,Value=Testing --region us-east-1
its giving me the following error
An error occurred (UnauthorizedOperation) when calling the CreateTags
operation: You are not authorized to perform this operation. Encoded
authorization failure message
You can use these credentials by invoking aws cli without any parameters related to credentials, it will try to pick up the creds from the instance profile. Your problem is not that you do not have the credentials but that you do not have permission to invoke CreateTags operation. As the error message says it is an authorization problem not an authentication one. You need to change the instance profile policy and include the capability to change instance tags.
More here:
https://docs.aws.amazon.com/codedeploy/latest/userguide/getting-started-create-iam-instance-profile.html
Check if your role allows you to create, list, delete tags on EC2 or if you require a custom policy attached with this role to allow these actions.
In summary, you should have:
{
"Effect": "Allow",
"Action": [
"ec2:CreateTags",
"ec2:DeleteTags"
]
}
The goal
I want to programmatically add an item to a table in my DynamoDB from my Elastic Beanstalk application, using code similar to:
Item item = new Item()
.withPrimaryKey(UserIdAttributeName, userId)
.withString(UserNameAttributeName, userName);
table.putItem(item);
The unexpected result
Logs show the following error message, with the [bold parts] being my edits:
User: arn:aws:sts::[iam id?]:assumed-role/aws-elasticbeanstalk-ec2-role/i-[some number] is not authorized to perform: dynamodb:PutItem on resource: arn:aws:dynamodb:us-west-2:[iam id?]:table/PiggyBanks (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException; Request ID: [the request id])
I am able to get the table just fine, but things go awry when PutItem is called.
The configuration
I created a new Elastic Beanstalk application. According to the documentation, this automatically assigns the application a new role, called:
aws-elasticbeanstalk-service-role
That same documentation indicates that I can add access to my database as follows:
Add permissions for additional services to the default service role in the IAM console.
So, I found the aws-elasticbeanstalk-service-role role and added to it the managed policy, AmazonDynamoDBFullAccess. This policy looks like the following, with additional actions removed for brevity:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"dynamodb:*",
[removed for brevity]
"lambda:DeleteFunction"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
This certainly looks like it should grant the access I need. And, indeed, the policy simulator verifies this. With the following parameters, the action is allowed:
Role: aws-elasticbeanstalk-service-role
Service: DynamoDB
Action: PutItem
Simulation Resource: [Pulled from the above log] arn:aws:dynamodb:us-west-2:[iam id?]:table/PiggyBanks
Update
In answer to the good question by filipebarretto, I instantiate the DynamoDB object as follows:
private static DynamoDB createDynamoDB() {
AmazonDynamoDBClient client = new AmazonDynamoDBClient();
client.setRegion(Region.getRegion(Regions.US_WEST_2));
DynamoDB result = new DynamoDB(client);
return result;
}
According to this documentation, this should be the way to go about it, because it is using the default credentials provider chain and, in turn, the instance profile credentials,
which exist within the instance metadata associated with the IAM role
for the EC2 instance.
[This option] in the default provider chain is available only when
running your application on an EC2 instance, but provides the greatest
ease of use and best security when working with EC2 instances.
Other things I tried
This related Stack Overflow question had an answer that indicated region might be the issue. I've tried tweaking the region with no additional success.
I have tried forcing the usage of the correct credentials using the following:
AmazonDynamoDBClient client = new AmazonDynamoDBClient(new InstanceProfileCredentialsProvider());
I have also tried creating an entirely new environment from within Elastic Beanstalk.
In conclusion
By the error in the log, it certainly looks like my Elastic Beanstalk application is assuming the correct role.
And, by the results of the policy simulator, it looks like the role should have permission to do exactly what I want to do.
So...please help!
Thank you!
Update the aws-elasticbeanstalk-ec2-role role, instead of the aws-elasticbeanstalk-service-role.
This salient documentation contains the key:
When you create an environment, AWS Elastic Beanstalk prompts you to provide two AWS Identity and Access Management (IAM) roles, a service role and an instance profile. The service role is assumed by Elastic Beanstalk to use other AWS services on your behalf. The instance profile is applied to the instances in your environment and allows them to upload logs to Amazon S3 and perform other tasks that vary depending on the environment type and platform.
In other words, one of these roles (-service-role) is used by the Beanstalk service itself, while the other (-ec2-role) is applied to the actual instance.
It's the latter that pertains to any permissions you need from within your application code.
To load your credentials, try:
InstanceProfileCredentialsProvider mInstanceProfileCredentialsProvider = new InstanceProfileCredentialsProvider();
AWSCredentials credentials = mInstanceProfileCredentialsProvider.getCredentials();
AmazonDynamoDBClient client = new AmazonDynamoDBClient(credentials);
or
AmazonDynamoDBClient client = new AmazonDynamoDBClient(new DefaultAWSCredentialsProviderChain());