We're moving a website to the AWS environment and running apache2 on an EC2. We were planning on using the AWS secrets manager to store some of the credentials such as the RDS (database) and email credentials. I also use environment variables in apache to store the AWS credentials, but since it is the AWS credentials which are used to retrieve the secrets, I was wondering if there was any way to run a script on apache start-up to use the aws-sdk to retrieve those secrets.
Or are there any other suggestions on how to do it? I can do it after the fact in the PHP code that needs such access, but I'm just exploring what is possible at the moment. It would be nice if some configuration options were loaded when apache starts.
Your two options seem to be either write a startup script that populates the env variables and then starts apache as a child process so it can read those variables, or modify the web application to read the secrets directly.
Reading the secrets directly in the application (possibly from some short lived cache) would allow you to rotate your DB creds without having to restart the application.
Also, if you are running on EC2 you do not need to populate env variables with the AWS creds. Just use roles for EC2. The AWS CLI and SDKs already know how to retrieve those credentials directly.
Related
How can I make the GO SDK fetch the access keys for AWS from the Instance Metadata Service (169.254.169.254) provided by AWS.
I checked the official AWS SDK for go documentation and there seems to be only ways of fetching the access keys from environment variables, but no credentials retriever from IMS.
How is this done in go?
I checked the official AWS SDK for go documentation and there seems to be only ways of fetching the access keys from environment variables, but no credentials retriever from IMS.
You just missed it. The Go SDK supports the instance metadata service as well as every other common credentials provider.
From https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html:
If you have configured your instance to use IAM roles, the SDK uses these credentials for your application automatically.
You don't have to do anything to configure this. It should just work. If you're having problems, make sure that you're not manually configuring some other credentials source.
Usually you don't have to do anything more than something like:
sess := session.Must(
session.NewSessionWithOptions(session.Options{
SharedConfigState: session.SharedConfigEnable,
}),
)
And with or without CLI configuration, metadata service, or environment variables, it should just work wherever you run it.
I have a Java program which needs to access Amazon S3 to put some files there. Please note that this Java program is running in my desktop (not in EC2). What's the best secure way to access Amazon S3 using credentials? Following are the ways I am aware of.
Using access token and secret
a. In sdk properties file
b. As environment variables
c. In command line system properties
d. Directly hard coding in program
Of course I'd prefer options b and c for security reasons.
Is there a role based permissions possible here? My understanding is that it's not possible since my Java program is running in an external machine which AWS doesn't know.
Any other method of access possible?
Thanks in advance.
The best way is to use the default provider chain, which means that the [DefaultCredentialsProvider] (https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/auth/credentials/DefaultCredentialsProvider.html) class will decide from where to take the credentials based on a specific hierarchy:
1. Java System Properties - aws.accessKeyId and aws.secretAccessKey
2. Environment Variables - AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
3. Web Identity Token credentials from system properties or environment variables
4. Credential profiles file at the default location (~/.aws/credentials) shared by all AWS SDKs and the AWS CLI
5. Credentials delivered through the Amazon EC2 container service if AWS_CONTAINER_CREDENTIALS_RELATIVE_URI" environment variable is set and security manager has permission to access the variable,
6. Instance profile credentials delivered through the Amazon EC2 metadata service
For local development the recommended way is to set up your credentials using the aws configure command and let the default provider chain take advantage of that.
Although environment variables may be a reasonable choice in some cases (and the default chain will be able to use them), please NEVER ever hardcode any credentials in your code!
Yes it is. We can assume a role using the AWS CLI:
aws sts assume-role --role-arn "arn:aws:iam::123456789012:role/example-role" --role-session-name AWSCLI-Session
This will provide a temporary AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN which can be provided to the application. The application will communicate with AWS services using the permissions provided by the assumed role.
Yes, there is another way if the goal is to access S3. We can use presigned urls.
When working with the AWS SDK for Java V2, refer to the AWS SDK for Java
Developer Guide V2. This developer guide contains a lot of information that answers questions like this.
To learn how credentials work, please refer to this topic:
Using credentials
All AWS Examples in Github assume credentials are loaded from the credential file. As explained in the docs, the credentials file is located in
Windows - C:\Users<yourUserName>.aws\credentials
Linux, macOS, Unix - ~/.aws/credentials
See this topic that will show you how to get up and running using the Amazon S3 API- including setting up your credentials.
Get started with the AWS SDK for Java 2.x
The Amazon S3 Java API has methods like pubObject that lets you place objects into an Amazon S3 bucket.
So it says on the github documentation here that
AWS Vault is a tool to securely store and access AWS credentials in a
development environment.
AWS Vault stores IAM credentials in your operating system's secure
keystore and then generates temporary credentials from those to expose
to your shell and applications. It's designed to be complementary to
the AWS CLI tools, and is aware of your
But what does this actually mean? As a developer does this mean to create a kind of lock to prevent anyone from using my code without the aws-vault profile? When should I use this technology? I want to know a bit more about it before I use it.
It actually doesn't have anything related to development.
While working with Amazon managed services we can take advantage of IAM roles but that doesn't work when you're doing it from our local environment or from some other Cloud VM like accessing a S3 bucket. It comes handy when you're doing a lot of work with AWS CLI or even writing terraform for your environment. It is just for a precaution so we don't expose or IAM credentials to external world (you will receive an abuse notification from Amazon whenever your keys are compromised). There are many other ways to make sure your keys don't get compromised like before pushing your code to a version control use git-secrets to make sure you don't push any sensitive information.
I'm currently using AWS S3 credentials inside my application's code in a file ignored by git, but every developer has that file so it can work with images.
I want to hide those credentials from the developers, but still have the ability to use S3 in the development environment.
What are the best practices on that?
Should I replace S3 usage to local files in the development environment?
Give each of your developers IAM accounts with their own API keys, set your application to read the API keys from environment variables rather than from a config file. When the code runs on EC2 instances, use IAM roles so that you don't need to use API keys at all. If you're using AWS SDKs it will use role credentials by default.
I have deployed the django application on aws . I want that application should be deployed by team as well. What is procedure to do this? I have searched a lot and almost spent couple of hours . Anyone has any answer or tutorial?
Can we share these keys ?
aws_access_key_id
aws_secret_access_key
No, the AWS access keys should be kept secret and not even stored under version control.
For deployment (i.e. the credentials needed to actually release the code - used by EB), you should use an aws profile. Add a ~/.aws/credentials file with
[myprofile]
aws_access_key_id=...
aws_secret_access_key=...
and then, on all eb commands use --profile. e.g.
eb create --profile myprofile
If your application requires other AWS services (e.g. RDS, S3, SQS), then you can use the same local profile for development (although I would recommend not requiring any other AWS for testing) by using then environment variable export AWS_PROFILE=myprofile. And then rely on AWS roles and policies for the production environment.
If you feel you need the secret keys as django settings, then consider using https://django-environ.readthedocs.org where you can keep all those secrets on a .env file that gets loaded by django. But again, this file should not be under version control.
You should also create IAM users for every person in your team, so each person has its own credentials, and you can more easily monitor or if needed, revoke credentials.