I am trying to connect to s3 bucket from the EC2 instance. The logging into ec2 instance is using private key, generated from pem file. Ideally I should have been able access the s3 bucket from here directly without passing access key and secret. But when i try to connect it is asking for above 2 things rahter than directly connecting and listing the contents.
The SSH key you generated from the pem file, and the AWS IAM account access key and secret key are two completely different things. The SSH key is used for initiating SSH connections with your EC2 server. The access key and secret key are from an AWS IAM account and are used for accessing the AWS API.
If you are trying to access S3 from your EC2 server the the prefered way is to assign an IAM role to the EC2 instance with the appropriate permissions, so that you don't have to deal with the IAM key credentials.
Related
I am trying to create a terraform script which is creating AWS IAM user with secret key and access key id. Same script is also creating AWS EC2.
now where I am getting stuck is in to set those keys in ~/.aws/credential file(like aws configure does) in EC2.
What you're trying to do is not the recommended solution. If you want to grant any access to your EC2 instances then do so via IAM Roles rather than keys. You have to attach a Role with permissions to your EC2 and it can call other services same like it would do from AWS profile (using keys).
Reference to how to achieve this in Terraform: https://skundunotes.com/2021/11/16/attach-iam-role-to-aws-ec2-instance-using-terraform/
I've created a node.js application which connects to DynamoDB. Everything is working fine locally Now I'm trying to setup on AWS servers.
First I've created DynamoDB tables from AWS DynamoDB console. It is working fine.
I've created a new role from IAM management console > Roles to access DynamoDB. And attached that role to EC2 instance.
But when I fire any aws dynamodb cli command, it gave me error to mention the region.
So I went to IAM management console > Users, and created an access key to my admin type user.
Now I'm login to EC2 CLI using ec2-user and aws configure with previously generated access key.
AWS Access Key ID [None]: ACCESS KEY
AWS Secret Access Key [None]: SECRET
Default region name [None]: us-east-1
Default output format [None]: json
But when I use following command aws dynamodb list-tables. It gives no output, no error.
As I commented, The main issue was outbound rules for attached security group. Here are the necessary things to do
Set a security group outbound rule to HTTPS
Setup Credentials
Create Access Key from IAM management console > Users.
SSH to EC2 instance.
Configure the credentials to EC2 instance using aws configure command or directly modify ~/.aws/credentials file.
Attach Role
Create Role from IAM management console > Roles. Select the role which is necessary to perform operation on AWS service. Eg AmazonDynamoDBFullAccess
Open VPC console and select the EC2 instance.
Attach the role from Actions menu
It is good, though optional, to create VPC endpoint. If you face UnauthorizedOperation error while creating endpoint, assign AmazonEC2FullAccess permission to the user from IAM console. Remove it later if you don't need it.
To use the AWS service from your application, find the relevant endpoint from this list.
It sounds like you are having problems connecting to DynamoDB because of the way you have configured your VPC.
There are some options but if you would prefer to keep your VPC isolated from the internet then you could enable VPC endpoints for DynamoDB. That way you can access DynamoDB from within your VPC without those connections going over the public internet.
There is a step-by-step guide for how to do that here: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/vpc-endpoints-dynamodb.html
Essentially, it involves the following steps:
you have to get the VPC id for the VPC where your EC2 instance is located
create a VPC endpoint for DynamoDB, specifying the VPC id and the regional dynamodb service name:
aws ec2 create-vpc-endpoint --service-name com.amazonaws.<region>.dynamodb --vpc-id <yourvpcid>
Using C#, I am able to download files from the bucket just by knowing the bucket name and the file key (filename).
The file and bucket are set up to not be accessible publicly.
Once I do
GetObjectRequest request = new GetObjectRequest
{
BucketName = bucketName,
Key = keyName
};
Even though I have not provided access key or the secret key, I still have access to the file content.
Is there a way to not allow this?
#akiva is partially correct.
If you are running this on an ec2 instance, if that instance has a 'IAM role' associated with it, and if that role has access to the bucket, the application can access the bucket without the application providing credentials.
On a regular machine, or even on an ec2 instance that does not have an associated IAM role, credentials are often stored in the users .aws subdirectory in a credentials file.
if you're connecting from an EC2 server it sometimes has a configuration file that stores the credentials globally for you
If your code is on EC2 machine, it fetches the access key and secret key stored in the hidden .aws directory. Also, if this EC2 instance has an IAM role with suitable permissions, it can fetch the contents from your S3 bucket very well.
You might need to check the role associated with your EC2 instance in order to prevent the accessibility to your S3 bucket.
I have the .PEM file for my AWS account. How to get AWS access key ID & secret access key to login to AWS CLI?
I am trying to configure my AWS CLI in windows using the command aws configure in command line. Its asking me parameters like
Access Key ID:
Secret Access Key:
I am not sure how to get these !! I only have .PEM file
I also don't have access to create one under the tab My Security Credentials->Users->Security Credentials
the .pem file allows you to connect to an ec2 instance that you have created.
the AWS access key id and AWS secret access key are information about your account and not linked to a specific instance. They are available when you create the account and after are not revealed to you
You can read this blog post from amazon to get more information why and how to get a new one. I'd always recommend to create a dedicated user to be used for the CLI even if you're the only one working on the account.
You need to contact your company's IT administrator, or whomever created the instance for you. They should be able to access your credentials and send it over to you. I had the same issue because I thought my .pem file was the correct credentials, but it was not. Once you obtain the access key ID and secret access key, you can then enter it in the following fields (it may have a slightly different terminology for these prompts, but you should get the idea):
access key ID:
secret access key:
region code: # This could be found on your management console (e.g. us-virginia-1)
Clarification: The .pem file contains the RSA private key, this is NOT what you need for configuring your AWS-CLI, the correct credentials will instead be referred to as 'access key ID' and 'secret access key', respectively.
I've signed up for a new AWS account and generated a new keypair. I've got the PEM file.
I start a new instance and log into it. I'm trying to do an 'aws configure' but its asking for aws_access_key_id and aws_secret_access_key. What are these? I've just got this PEM file.
AWS Security Credentials
When you interact with AWS, you use AWS security credentials to verify who you are and whether you have permission to access the resources that you are requesting. In other words, security credentials are used to authenticate and authorize calls that you make to AWS.
Types of Security Credentials
Access keys (access key ID and secret access key)
Access keys consist of an access key ID (like AKIAIOSFODNN7EXAMPLE) and a secret access key (like wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY). You use access keys to sign programmatic requests that you make to AWS whether you're using the AWS SDK, REST, or Query APIs.... Access keys are also used with command line interfaces (CLIs).
Finally, under How Do I Get Security Credentials?
Access keys (access key ID and secret access key)
... You can create new access keys for the account by going to the Security Credentials page. In the Access Keys section, click Create New Access Key.