I want to deploy a node application on a google cloud compute engine micro instance from a source control repo.
As part of this deployment I want to use KMS to store database credentials rather than having them in my source control. To get the credentials from KMS I need to authenticate on the instance with GCLOUD in the first place.
Is it safe to just install the GCloud CLI as part of a startup script and let the default service account handle the authentication? Then use this to pull in the decrypted details and save them to a file?
The docs walkthrough development examples, but I've not found anything about how this should work in production, especially as I obviously don't want to store the GCloud credentials in source control either.
Yes, this is exactly what we recommend: use the default service account to authenticate to KMS and decrypt a file with the credentials in it. You can store the resulting data in a file, but I usually either pipe it directly to the service that needs it or put it in tmpfs so it's only stored in RAM.
You can check the encrypted credentials file into your source repository, store it in Google Cloud Storage, or elsewhere. (You create the encrypted file by using a different account, such as your personal account or another service account, which has wrap but not unwrap access on the KMS key, to encrypt the credentials file.)
If you use this method, you have a clean line of control:
Your administrative user authentication gates the ability to run code as the trusted service account.
Only that service account can decrypt the credentials.
There is no need to store a secret in cleartext anywhere
Thank you for using Google Cloud KMS!
Related
We need to create gcp resources with terraform, but we are stuck at the terraform init stage while terraform tries to authenticate to gcp. We have already configured our backend and obtained our service account key but minifying (removing the extra lines in credential json file) the credential json and exporting to GOOGLE_CREDENTIALS, doesn't work. How are you setting this value?
If you are in a local and controlled environment you can use GOOGLE_APPLICATION_CREDENTIALS and set it with the path to the JSON key file. But as discussed key files are bad practices security wise. An alternative is to authenticate using gcloud auth application-default login and you dont have to deal with key files.
Another alternative is to use Google Cloud Shell which is already setup with the credentials of the authorised user opening the session.
Finally for automated pipeline you can use Google Cloud Build where processes will be run using the authentication and the authorisation of the service account used by Cloud Build
Currently we put the google cloud credentials in json file and set it via GOOGLE_APPLICATION_CREDENTIALS environment variable.
But we do not want to hardcode the private key in the json file due to security reasons. We want to put the private key in Azure key vault (yes we use azure key Vault), is there a way to provide the credentials programatically to GCP. so that i can read azure key vault and provide the private key via code. I tried to check and use GoogleCredentials and googles DefaultCredentialsProvider etc classes, but i could not find a proper example.
Note: The google credentials type is Service Credential
Any help is much appreciated.
Store the service account JSON base64 encoded in Azure Key Vault as a string.
Read the string back from Key Vault and base64 decode back to a JSON string.
Load the JSON string using one of the SDK member methods with names similar to from_service_account_info() (Python SDK).
John's answer is the correct one if you want to load a secret from Azure vault.
However, if you need a service account credential in Google Cloud environment, you don't need a service account key file, you can use the automatically loaded service account into the metadata server
In almost all Google Cloud service, you can customize the service account to use. In the worse case, you need to use the default service account for the context, and grant it the correct permissions.
Service account key file use isn't a good practice on Google Cloud product. It's a long lived credential, and you have to rotate yourselves, keep it secrets,...
I am writing an application where I have to upload media files to GCS. So, I created a storage bucket and also created a service account which is being used by the application to put and get images from the bucket. To access this service account from the application I had to generate a private key as a JSON file.
I am tested my code and it is working fine. Now, I want to push this code to my Github repository but I don't want this service account key to be in Github.
How do I manage to keep this service account key secret, yet all my fellow colleagues should be able to use it.
I am going to put my application on GCP Container Instance and I want it to work there as well.
As I understand, if your application works from inside the GCP and use some custom service account, you might not need any private keys (as json files) at all.
The custom service account, which is used by your application, should get relevant IAM roles/permissions on the correspondent GCS bucket. And that's all you might need to do.
You can assign those IAM roles/permissions either manually (through UI console), or using CLI commands, or as part of your deployment CI/CD pipeline.
Is it possible to do local development without using google service account key in minikube.
Currently I have a service account key which I use to authenticate google services inside the pod in minikube.
I want to avoid using service account key and instead use IAM role.
Iam User - XXX#XX.com
I am given required role/permission to this iam user.
After using gcloud auth login, I can access the google services. Is it possible to do similar way in k8s/minikube pods.
I think that you can cheat. Use it only for development environment, never in production
Firstly, locate your own user credentials created with the gcloud auth application-default login. The created file is
on linux ~/.config/gcloud/application_default_credentials.json
on windows %appdata%\gcloud\application_default_credentials.json
Then mount this file (or copy it) into minikube
You can define your GOOGLE_APPLICATION_CREDENTIALS env var to reference this file.
That's all, your credential will be used! Be careful, there is some limitation, I wrote an article on this
I think you are mixing things up. You can never use a key instead of a role. In most cases, you need both of them. You need a key to authenticate to the Google Cloud Platform. And you need a certain IAM role to access services within GCP. Authentication means confirming your own identity, whereas authorization means being allowed access to the system.
In your specific case, I think you are referring to the process of letting your application/code use your own identity to authentication to the Google Cloud Platform. There are 2 possibilities here:
Download a service account key file, which is prone to security leaks, because those key files are not rotated by themselves.
As #guillaume blaquiere explains below, you could also generate a key file using your own identity. The specifics about this are well explained here and here. For local development, this is preferred over the other option.
If you want to know how your SDK works with key files, I would recommend you take a look inside the SDK for the programming language you are using. There is always be a reference to GOOGLE_APPLICATION_CREDENTIALS. This is the location of the key file you are using.
I have a CLI tool that interacts with Google KMS. In order for it to work, I fetch the user credentials as a JSON file which is stored on disk. Now a new requirement came along. I need to make a web app out of this CLI tool. The web app will be protected via Google Cloud IAP. Question is, how do I run the CLI tool on behalf of the authenticated user?
You don't. Better use a service-account and assign the required role. That service account still could have domain-wide delegation of rights (able to impersonate just any user, which is known).
Running CLI tools from a web-application probably also could/should be avoided. Iit might be better to convert his CLI tool into a Cloud Function and then call it via HTTP trigger, from within the web-application (so that access to the service account is limited as far as possible).
This might also be something to reconsider, security-wise:
I fetch the user credentials as a JSON file which is stored on disk.
Even if it might have been required, with a service-account it wouldn't.