We have usecase of taking input which includes password from user and pass it to on EC2 instance. From with in Ec2 instance we hit the URL - http://169.254.169.254/latest/user-data/ and get the userdata and set appropriate passwords.
The issue is user data is visible by AWS CLI tool:
aws ec2 describe-instance-attribute --instance-id --attribute userData --output text --query "UserData.Value" | base64 --decode
This imposes huge security risk.
Whats the best way to send sensitive / secret data ?
I tried creating a key-pair, which creates the private key on local instance and public key on EC2. What would be right way to encrypt / decrypt using PowerShell and fetch it back in EC2?
The suggested approach would be to store any secrets in an external source.
AWS has a service for storing secrets, Secrets Manager. By using this service you would create a secret containing the secrets that your instance will need to access in its user data. Then give your instance an IAM role with privileges to get the secret value, via the AWS CLI.
Alternatively you could also make use of the AWS SSM Parameter Store service, storing the secrets as a SecureString type. This would work similar to secrets manager with you retrieving the secret via the AWS CLI and then using it in your script.
There are also third party solutions such as Hashicorp Vault that provide similar functionality if you do not want to store your secrets in an AWS solution.
Related
I have bash script to run as user data script when launching EC2 instance. For that I need to pass external API access key id and secret key. I don't want to store these keys in my user data scripts as it is visible in plaintext. Is there any way that I can store this keys in somewhere such as AWS Secret Manager and use that in user data scripts?
I would suggest either storing it in Secrets Manager or SSM Parameter Store.
You would need to use the CLI in your userdata script to retrieve the value.
For SSM you would retrieve the secret by using the get-parameter function.
secret=$(aws ssm get-parameter --name "MyStringParameter")
For Secrets Manager you would retrieve the secret using the get-secret-value function.
secret=$(aws secretsmanager get-secret-value --secret-id MyTestDatabaseSecret)
Then in your bash script when you want to reference it you would just need to use the variable $secret to actually replace with your secret.
If you decide to use either of these you will need to ensure EC2 instance has an IAM role attached to the instance with the correct policy to apply the permissions you require.
Alternatively if this is a process that happens frequently (autoscaled instance for example) then you should take a look at configuring the base server image (AMI) ahead of time and then referencing this as the source AMI.
With tools such as Ansible, Chef and Puppet you could provision the base image with your secret which would replace any need to do anything in the UserData as it would be available ahead of time.
Usually you can store such secrets in AWS Systems Manager Parameter Store which is free, unlike AWS Secret Manager:
AWS Systems Manager Parameter Store provides secure, hierarchical storage for configuration data management and secrets management. You can store data such as passwords, database strings, Amazon Machine Image (AMI) IDs, and license codes as parameter values.
To use that in your UserData, the instance role has to be set with permissions to access the Parameter Store. Then in your UserData you can use aws cli get-parameter to get the value of your secrets.
I am using AWS Secret Manager Service to retrieve some confidential information like SMTP details or connection strings. However, to get secret value from AWS Secret Manager Service it seems like we need to pass the Access key and secret key apart from which secret we want to retrieve. So I am maintaining those values in config file.
public AwsSecretManagerService(IOptions<AwsAppSettings> settings)
{
awsAppSettings = settings.Value;
amazonSecretsManagerClient = new AmazonSecretsManagerClient
(awsAppSettings.Accesskey, awsAppSettings.SecretKey, RegionEndpoint.GetBySystemName(awsAppSettings.Region));
}
public async Task<SecretValueResponse> GetSecretValueAsync(SecretValueRequest secretValueRequest)
{
return _mapper.Map<SecretValueResponse>(await amazonSecretsManagerClient.GetSecretValueAsync(_mapper.Map<GetSecretValueRequest>(secretValueRequest)));
}
So I am thinking I am kind of defeating the whole purpose of using secret manager by maintaining the AWS credentials in app settings file. I am wondering what is the right way to do this
It is not a good practice to pass or add AWS credentials of an IAM User (access key and secret access key) in the code.
Instead, don't pass it and update your code as follows:
amazonSecretsManagerClient = new AmazonSecretsManagerClient
(RegionEndpoint.GetBySystemName(awsAppSettings.Region));
Question: Then how would it access the AWS services?
Answer: If you are going to execute your code on your local system, install and configure AWS CLI instead of passing AWS credentials via CLI or Terminal, it will use those AWS configured credentials to access the AWS services.
Reference for AWS CLI Installation: Installing the AWS CLI
Reference for AWS CLI Configuration: Configuring the AWS CLI
If you are going to execute your code on an AWS service (e.g., EC2 instance), attach an IAM role with that AWS resource (e.g., EC2 instance) having sufficient permissions, it will use that IAM role to access the AWS services.
For different AWS services, I need different IAM users to secure the access control. Sometimes, I even need to use different IAM user credentials within a single project in a EC2 instance. What's the proper way to manage this and how I can deploy/attach these IAM user credentials to a single EC2 instance?
While I fully agree with accepted answer that using static credentials is one way of solving this problem, I would like to suggest some improvements over it (and proposed Secrets Manager).
What I would advise as architectural step forward to achieve full isolation of credentials, having them dynamic, and not stored in central place (Secrets Manager proposed above) is dockerizing application and running on AWS Elastic Container Service (ECS). This way you can assign different IAM role to different ECS Tasks.
Benefits over Secrets Manager solution
- use case of someone tampering with credentials in Secrets Manager is fully avoided, as credentials are of dynamic nature (temporary, and automatically assumed through SDKs)
Credentials are managed on AWS side for you
Only ECS Service can assume this IAM role, meaning you can't have actual person stealing the credentials, or developer connecting to production environment from his local machine with this credentials.
AWS Official Documentation for Task Roles
The normal way to provide credentials to applications running on an Amazon EC2 instance is to assign an IAM Role to the instance. Temporary credentials associated with the role when then be provided via Instance Metadata. The AWS SDKs will automatically use these credentials.
However, this only works for one set of credentials. If you wish to use more than one credential, you will need to provide the credentials in a credentials file.
The AWS credentials file can contain multiple profiles, eg:
[default]
aws_access_key_id = AKIAaaaaa
aws_secret_access_key = abcdefg
[user2]
aws_access_key_id = AKIAbbbb
aws_secret_access_key = xyzzzy
As a convenience, this can also be configured via the AWS CLI:
$ aws configure --profile user2
AWS Access Key ID [None]: AKIAbbbb
AWS Secret Access Key [None]: xyzzy
Default region name [None]: us-east-1
Default output format [None]: text
The profile to use can be set via an Environment Variable:
Linux: export AWS_PROFILE="user2"
Windows: set AWS_PROFILE="user2"
Alternatively, when calling AWS services via an SDK, simply specify the Profile to use. Here is an example with Python from Credentials — Boto 3 documentation:
session = boto3.Session(profile_name='user2')
# Any clients created from this session will use credentials
# from the [user2] section of ~/.aws/credentials.
dev_s3_client = session.client('s3')
There is an equivalent capability in the SDKs for other languages, too.
i have created an aws account, launched ec2 instance and created buckets in s3. Also i have installed python, boto3 and aws cli. But i'm stuck on connecting python with aws step.
The first and foremost thing that you need to check is whether your EC2 instance has permissions to access the S3 bucket. This can be done in 2 ways:
Store the credentials in the EC2 instance (insecure)
Assign IAM roles to the EC2 instance that has S3 read and write permissions (secure)
In order to assign a role to your instance, follow this guide.
Once your permissions are set up, you can either use the AWS CLI or BOTO3 to access S3 from your EC2 instance.
1: If you are asking how to establish a connection for running your AWS-python codes then you must follow these steps on terminal:
aws configure (this ill ask you credentials which you will find in the .CSV file that is created initially)
Provide the credentials and try to run the code
For ex:
$ aws configure
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-west-2
Default output format [None]: json
2: If your question is how you will use the boto3 API calls to run AWS functions then this might help you:If you are using boto3 SDK then you can make use of low-level clients and higher-level.
ec2 = boto3.resource('ec2')
client = boto3.client('ec2')
You can follo this link for more detailed info : http://boto3.readthedocs.io/en/latest/reference/services/ec2.html
I am mounting an AWS S3 bucket as a filesystem using s3fs-fuse. It requires a file which contains AWS Access Key Id and AWS Secret Access Key.
How do I avoid the access using this file? And instead use AWS IAM roles?
As per Fuse Over Amazon document, you can specify the credentials using 4 methods. If you don't want to use a file, then you can set AWSACCESSKEYID and AWSSECRETACCESSKEY environment variables.
Also, if your goal is to use AWS IAM instance profile, then you need to run your s3fs-fuse from an EC2 instance. In that case, you don't have to set these credential files/environment variables. This is because while creating the instance, if you attach the instance role and policy, the EC2 instance will get the credentials at boot time. Please see the section 'Using Instance Profiles' in page 190 of AWS IAM User Guide
there is an argument -o iam_role=--- which helps you to avoid AccessKey and SecretAccessKey
The Full steps to configure this is given below
https://www.nxtcloud.io/mount-s3-bucket-on-ec2-using-s3fs-and-iam-role/