I'm trying to develop a Django app on GAE, and using CloudBuild for CI/CD. I'm wondering what's the best way to pass secrets to my app (DB credentials, etc).
I was able to follow instructions at https://cloud.google.com/cloud-build/docs/securing-builds/use-encrypted-secrets-credentials to read the secret from in my build step, and pass it in to my app as an environment variable. It's a bit hacky, but it works:
- name: gcr.io/cloud-builders/gcloud
entrypoint: 'bash'
args:
- '-c'
- |
TEST_PW=$(gcloud secrets versions access latest --secret=test-key)
echo "TEST_PASSWORD=$${TEST_PW}" >> env_vars
unset TEST_PW
However, I'm not sure if this practice is safe. I dumped the env variables in running in my app (using print(dict(os.environ)) and the only sensitive values there are the secrets I passed in (all other GAE app related values are non-sensitive data).
So questions:
1) Is storing secrets in env variables safe in an app in AppEngine, i.e. can they be stolen by "somehow" dumping them through print(dict(os.environ))?
2) Or is the better option to fetch them from Secret Manager in Django (for e.g. in settings.py)? (I'm worried about restarts or version switches here, and if they'll affect this option)
3) Or is there an even better option?
Thanks.
The security issue with what you are doing is not the environment variable itself, but the fact that you are storing the secret's plain decrypted value in it, making it accessible by the os.environ command while your instance is running.
A simpler solution would be to dump that sensitive information to a file and store it on a Cloud Storage Bucket only your app engine's service account has access to, like this:
TEST_PW=$(gcloud secrets versions access latest --secret=test-key)
echo "TEST_PASSWORD=$${TEST_PW}" >> [YOUR_FILE_URL]
unset TEST_PW
If you want to keep using environment variables, you can do it by using Cloud KMS to keep data encrypted, you can find a how to here, which is a different section of the same documentation you shared on your question.
Related
I am beginner to aws.
aws keys stored in pc in folder "C:\Users\USERNAME.aws\credentials" is simple text file.
Q1) Why it is stored in easy readable format?
Q2) How to protect these credentials or is there any way to store it in encrypted way?
Q2) One option is to create an environment variable within your code’s operating environment to store the key. Environment variables are managed by the environment’s operating system and can be accessed via system libraries or the AWS SDK for your preferred programming language.
Q1) Thats they way it stores when you run aws configure post awscli installation.
On ways to secure it more :
Follow AccessKey Rotation practice to make sure that even if ur access key falls into wrong hands rotating it and creating a new accesskey would protect from any mishaps.
Use AWS Secret Manager to store your secrets which also gives you options to rotate secret values as well.
The user folder is generally considered to be private to the user. In typical OS setup, the permissions would be set, so that only the user has access to their home directory. Of course anyone who gains access to that folder also has access to the keys, but that's not any different for other common scenarios like storing ssh keys in so called 'hidden' .ssh/ folder
In any case, if you are not comfortable with that, the other option is to store them where ever you feel safe, then retrieve them and temporarily add them to your user environment profile.
The environment variables are documented here: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html
Why do people put a .env file to store all their secrets in a server? If someone hacks it, isn't the .env equally accessible as all the other files? Thanks!
You are correct that storing environmental secrets in a .env file poses a risk of plain text secrets being exposed to a third party if they gained access to raw code.
Just like other areas with sensitive material there are ways to get around this, generally an approach that people might take it to use a secrets management system which instead replaces any secrets values from a .env file to be accessed via a validated request.
AWS supports a couple of official services that can do this:
Secrets Manager - This service is specifically built for this purpose, you define a secret and give it either a string or JSON value that is then retrieved via a call using the SDK. All values are encrypted using a KMS key.
Systems Manager Parameter Store - Similar to secrets manager, you provide a key name and give it a value. It supports both unencrypted and encrypted values (use SecureString type).
In addition there are other services such as Hashicorp Vault that provide similar functionality.
For environmental configuration a .env file can still be appropriate i.e. enable this feature flag but if you want to try and reduce the blast radius of your application then storing secrets outside a plain text file will help to reduce this risk.
That is not the main reason for using environment variables. However, it is secure enough for saving secret values too especially when they’re combined with hashing methods.
Environment variables are most useful in the actual production level of programming. Your application must have different environments to run upon. Development: that your host is local and as a developer you need to test your code and set the debug variable to true to get stateful errors which is not something you want on the production environment. Production: that your host is your domain or server IP and you need different middleware than of the development stage. There are also staging and test environments for bigger projects. There could be a lot of things that should be handled differently on different environments: database is a great example. Besides, environment variables are useful for when there is more than one person working with the code base and people can configure the project based on their machine/OS using environment variables.
Imagine you create a docker-compose.yml with Django and a bunch of code and use an environment variable to configure the SECRET_KEY in settings.py.
If you distribute that Docker image you won't share the SECRET_KEY.
What SECRET_KEY should someone use when they deploy your Docker image?
They can't make up their own SECRET_KEY right?
According to the documentation, changing the secret key will invalidate:
All sessions if you are using any other session backend than django.contrib.sessions.backends.cache, or are using the default get_session_auth_hash().
All messages if you are using CookieStorage or FallbackStorage.
All PasswordResetView tokens.
Any usage of cryptographic signing, unless a different key is provided.
What's the best way to 'renerate' a secret key after deploying a Docker container with a bunch of code that you created?
(I have searched and searched but feel like I'm searching for the wrong things or completely missed something :-)
Everybody who deploys the service independently should have their own SECRET_KEY. (You actively do not want the things you describe to be shared across installations: if I’ve logged into my copy of your service, I shouldn’t be able to reuse my session cookie on your copy.) A command I typically use for this is
dd if=/dev/urandom bs=60 count=1 | base64
which will generate an 80-character high-quality random key.
The corollary to this is that you can’t distribute encrypted data with your image. That’s usually not a problem (it is difficult to distribute a prepopulated relational database in Docker) and if you do this by running database migration and seed jobs at first startup, they should be able to use whatever key you set when you do the installation.
This solution is platform agnostic because it uses the original Django key generator.
from django.core.management.utils import get_random_secret_key
get_random_secret_key()
It can be used standalone without initializing a Django project.
I have set up a Symfony 4 project and deployed it to AWS.
The credentials for the database connection are available via the server globals. Is there a way to access those variables directly from the .env file? If not, what is the best approach to connect to the database without using hardcoded credentials?
Basically I want to define my connection variables like this so that I can assign them to the DATABASE_URL.
RDS_HOSTNAME=$_SERVER['RDS_HOSTNAME']
Yes, you could access those variables directly from the .env file. As below. With this way, you are supposed to install Dotenv Bundle
RDS_HOSTNAME=getenv("RDS_HOSTNAME")
BUT, this approach is the best way on development. While you are running on production, they recommend environment variable approach.
Symfony Dotenv can be used in any environment of your application: development, testing, staging and even production. However, in production it's recommended to configure real environment variables to avoid the performance overhead of parsing the .env file for every request.
https://symfony.com/doc/current/components/dotenv.html
I remember being told to store AWS key into a bash**file (something named like that, can't remember exactly) for security reason, but now I forgot how to access that bash**file.
It should be ~/.bash_profile. Open a terminal window and type.
vi ~/.bash_profile.
To prevent committing sensitive application keys/data to your code, and to provide key access to programs, you should store app keys/sensitive information in environment variables. Environment variables are similar to variables in computer programs, except they exist system-wide in Linux and Windows.
In Linux, you can store those keys in the ~/.bash_profile, so they are available in the environment to command line programs.
nano ~/.bash_profile
in that file, add the following:
export AWS_ACCESS_KEY_ID= *ACCESS_KEY*
export AWS_SECRET_ACCESS_KEY= *SECRET_KEY*
Once saved, you’ll need to source it for the environment variable to work in your current session:
source ~/.bash_profile
In any new session, the environment variables will be loaded automatically.
Please note there are new and more preferred ways to store AWS credentials.
The AWS SDK team has recently made some changes that make it more
convenient, more consistent, and easier to specify credentials for the
SDKs in a more secure way.
Instead of keeping AWS credentials in environment variables, you can now put credentials into a single file that’s in a central location. The default location is this:
~/.aws/credentials (Linux/Mac)
See https://aws.amazon.com/blogs/security/a-new-and-standardized-way-to-manage-credentials-in-the-aws-sdks/