We have GCP account credentials(username/password). We have installed gcloud CLI on the Amazon Linux EC2 machine. We would like to create a script that would auto-login to the GCP account and do the below things sequentially using gcloud CLI.
Login to the GCP account.
Create Project and specify a meaningful project-id.
Create a service account with a meaningful ID.
Assign the owner role to the service account.
Create and download a new JSON key.
Please help us to achieve this
You should use a Service Account not a User (username|password) for automation. The Service Account should be suitably permissioned so that it can create Projects and Service Accounts.
I was unable to find a source for this (but it used to be that?) Google monitors User Accounts for apparent use of automation (e.g. for bots) and these accounts may be disabled.
Related
I want to automate the creation of development servers within my org's GCP account that are tied to an individual developer.
I intend to script the copying of relevant credentials (think crypto key for GitHub). How do I lockdown the VM so that only a single user can access it. GCP has options like block-project-ssh-keys but we've found that all users are still able to login using gcloud compute ssh.
I fear this may require a really complex IAM configuration.
Is there an easy way to accomplish this? (IAM or otherwise)
Has anyone seen a project that has automated this or something similar to this? (IAM provisioning of a new tag/group/etc. using a script when a new user joins the org)
Enable OS Login on the project, ideally by using an organizational policy constraint. OS Login automatically disables metadata-based SSH keys.
Grant OS Login roles individually per VM and user so that each user can only login to their own VM.
Either don't attach service accounts to the VMs at all, or create dedicated service accounts per VM. Then grant users the Service Account user role on "their" service account only.
Don't grant any Compute* roles beyond Compute Viewer on the project. In particular, don't grant Compute Admin or Compute Instance Admin, or any other role that lets users modify instance metadata (because that would let them alter startup and shutdown scripts).
Optionally:
Grant the users Compute Viewer on the project. That ensures that they can list VMs in the Cloud Console.
Only allow SSH access via IAP TCP forwarding, and grant the IAP-secured tunnel user role on a per-VM, per-user basis.
Service account "abcdefc-compute#developer.gserviceaccount.com" does not exist.
I am trying to create a kubernetes cluster but GCP gives me the error above.
I checked for the account name in service account but could not find it, rather I have
'ayushaccount#abcdef.iam.gserviceaccount.com'.
I tried to create another service account with this email "abcdefc-compute#developer.gserviceaccount.com" but it does not allow me to create.
I am new to GCP and I do not know how to solve this problem. All I am looking for to create a kubernetes cluster in GCP.
Looks like you are missing the default service account for your GCP project.
You have two options:
(re)create the default service account
when creating your GKE cluster, under NODE POOLS, go to default-pool->Security and for Service account, select one the one which exists.
If you want to (re)create the default service account, you can disable/enable the Google Compute Engine API via the console or run gcloud services enable compute.googleapis.com from Cloud Shell or from the command line on your workstation.
I created a service account mycustomsa#myproject.iam.gserviceaccount.com.
Following the GCP best practices, I would like to use it in order to run a GCE VM named instance-1 (not yet created).
This VM has to be able to write logs and metrics for Stackdriver.
I identified:
roles/monitoring.metricWriter
roles/logging.logWriter
However:
Do you advise any additional role I should use? (i.e. instance admin)
How should I setup the IAM policy binding at project level to restrict the usage of this service account just for GCE and instance-1?
For writing logs and metrics on Stackdriver those roles are appropriate, you need to define what kind of activities the instance will be doing. However as John pointed in his comment, using a conditional role binding 1 might be useful as they can be added to new or existing IAM policies to further control access to Google Cloud resources.
As for the best practices on SA, I would recommend to make the SA as secure as possible with the following:
-Specify who can act as service accounts. Users who are Service Account Users for a service account can indirectly access all the resources the service account has access to. Therefore, be cautious when granting the serviceAccountUser role to a user.
-Grant the service account only the minimum set of permissions required to achieve their goal. Learn about granting roles to all types of members, including service accounts.
-Create service accounts for each service with only the permissions required for that service.
-Use the display name of a service account to keep track of the service accounts. When you create a service account, populate its display name with the purpose of the service account.
-Define a naming convention for your service accounts.
-Implement processes to automate the rotation of user-managed service account keys.
-Take advantage of the IAM service account API to implement key rotation.
-Audit service accounts and keys using either the serviceAccount.keys.list() method or the Logs Viewer page in the console.
-Do not delete service accounts that are in use by running instances on App Engine or Compute Engine unless you want those applications to lose access to the service account.
My personal account is an admin in my gcp project.
If I want to use one of the service accounts I have created (from my local laptop) I do this:
gcloud auth activate-service-account --key-file=some-service-account.json
But I wonder, if I already have my own admin account active, is there a way to just assume a service account without the key? Can GCP use my current creds to give me access to assume that service account?
If so this also makes me wonder if I can use service accounts applied to GCE instances the same way. So I can attach a service account to a GCE instance that gives it access to assume other service accounts.
I think what you're looking for is "impersonation". You need roles like iam.serviceAccountUser to do this. Refer to these docs and articles:
https://cloud.google.com/iam/docs/service-accounts#the_service_account_user_role
https://medium.com/google-cloud/using-serviceaccountactor-iam-role-for-account-impersonation-on-google-cloud-platform-a9e7118480ed
https://medium.com/google-cloud/impersonating-users-with-google-cloud-platform-service-accounts-ba762db09092
I am trying to list the jobs running in big query for many projects using a user account which is having owner access on the gcp projects. We are using python APIs and all this process is running on a VM instance. But listing the job is failing because default service account for the VM instance doesn't have permission for other projects.
What I am unable to understand is, why using service account when the user have all the access. We don't want to create a service account with owner access, So is there anyway that we can list the BQ jobs only with our own account and not using service account.
This is the python code I am using for listing the job:
from google.cloud import bigquery
import pandas as pd
client = bigquery.Client(project=<project_ID>)
job_list = client.list_jobs(project=<project_ID>,max_results=100000, state_filter='running', all_users=True)
I tried giving the credentials with json file using below command, But that also was not helpful.
client = bigquery.Client.from_service_account_json("0874ee00257b.json")
Because by default, the GCE instance (VM) is authenticated using the service account it has been given access to when it was created (default service account). That VM & service account is linked only to the project in which it has been created, and not to a user. This makes sense when you think about it. You wouldn't want VMs (or any services on GCP in fact) authenticated/tied to an individual user. This would be bad practice e.g. what is that user leaves the company and their account is deleted.
So is there anyway that we can list the BQ jobs only with our own account and not using service account.
So, back to your actual question. Yes, but I wouldn't recommend doing this if it's something you intend to deploy and productionize/operationalize.
SSH into the VM
Run gcloud auth login
Follow the prompts
Note: when you do this, Google will even tell you it's not recommended and you should use service accounts instead:
Some more info here: https://cloud.google.com/sdk/docs/authorizing