I'm trying to setup a GKE cluster and I want to enable all permissions to other services (since apparently you can't change the permissions after the cluster has been created). This ought to be straight-forward but either I'm doing something wrong or something is broken. I select the following for my project access:
But when the cluster is created a see this:
I.e. everything is disabled. Why is this? How do I set the permissions?
There was a bug in the UI that was showing that all scopes were disabled. I just created a new cluster and the UI is showing the correct scopes.
If this happens again, you can also see the scopes that are enabled on your VMs using the command line by running gcloud container clusters describe NAME --zone=europe-west1-c and looking at the scopes under oauthScopes:.
Related
We are in process of improving the IAM roles in our project and we need to enable dev team to only resize their cluster to save the cost.
We are struggling to get the exact set of permissions needed to enable user to only scale up and scale down cluster nodes (i.e. resizing). We referred below GCP IAM documentation but it didn't help either to get this information.
https://cloud.google.com/iam/docs/permissions-reference
Currently, we have given below set of permissions(some of them may not required) however we are not able to do cluster resizing with this. And one more issue is GKE does not give any permission error, we see the "Node Pool Resized Successfully" notification but nodepool size doesn't change.
Is there any documentation or link which has the mapping of set of permissions vs user activity type of mapping for GCP IAM.
The GKE cluster will be created with the permissions that is set on the 'Access scopes' section in the 'Advanced edit' tab. So only the APIs with the access enabled in this section will be shown as enabled. These permissions denote the type and level of API access granted to the VM in the node pool. Scopes inform the access level your cluster nodes will have to specific GCP services as a whole. Please see this link for more information about accesss scopes.
In the tab of 'Create a Kubernetes cluster', click 'Advanced edit'. Then you will see another tab called 'Edit node pool' pops up with more options. If you click 'Set access for each API', you will see the option to set these permissions.
'Permissions' are defined when the cluster is created. You can not edit it directly on the cluster after the creation. You may want to create a new cluster with appropriate permissions or create a new Node Pool with the new scopes you need and then delete your old 'default' Node Pool as specified in this link .
When you add or remove nodes in your cluster, Google Kubernetes Engine (GKE) adds or removes the associated virtual machine (VM) instances from the underlying Compute Engine Managed Instance Groups (MIGs) provisioned for your node pools.
Please see this link for more information about resizing the cluster.
I was happily deploying to Kubernetes Engine for a while, but while working on an integrated cloud container builder pipeline, I started getting into trouble.
I don't know what changed. I can not deploy to kubernetes anymore, even in ways I did before without cloud builder.
The pods rollout process gives an error indicating that it is unable to pull from the registry. Which seems weird because the images exist (I can pull them using cli) and I granted all possibly related permissions to my user and the cloud builder service account.
I get the error ImagePullBackOff and see this in the pod events:
Failed to pull image
"gcr.io/my-project/backend:f4711979-eaab-4de1-afd8-d2e37eaeb988":
rpc error: code = Unknown desc = unauthorized: authentication required
What's going on? Who needs authorization, and for what?
In my case, my cluster didn't have the Storage read permission, which is necessary for GKE to pull an image from GCR.
My cluster didn't have proper permissions because I created the cluster through terraform and didn't include the node_config.oauth_scopes block. When creating a cluster through the console, the Storage read permission is added by default.
The credentials in my project somehow got messed up. I solved the problem by re-initializing a few APIs including Kubernetes Engine, Deployment Manager and Container Builder.
First time I tried this I didn't succeed, because to disable something you have to disable first all the APIs that depend on it. If you do this via the GCloud web UI then you'll likely see a list of services that are not all available for disabling in the UI.
I learned that using the gcloud CLI you can list all APIs of your project and disable everything properly.
Things worked after that.
The reason I knew things were messed up, is because I had a copy of the same things as a production environment, and there these problems did not exist. The development environment had a lot of iterations and messing around with credentials, so somewhere things got corrupted.
These are some examples of useful commands:
gcloud projects get-iam-policy $PROJECT_ID
gcloud services disable container.googleapis.com --verbosity=debug
gcloud services enable container.googleapis.com
More info here, including how to restore service account credentials.
I want to create GKE clusters by gcloud command. But I cannot solve this error:
$ gcloud container clusters create myproject --machine-type=n1-standard1# --zone=asia-northeast1-a
ERROR: (gcloud.container.clusters.create) ResponseError: code=403, message=Google
Compute Engine: Required 'compute.networks.get' permission for
'projects/myproject/global/networks/default'
cloud account linked to my gmail is owner of the project and relative powers, so I anticipate that there is no problem about permissions.
When you create a cluster though $ gcloud container clusters create command you should keep in mind that there are hundreds of operations hidden.
When you have the owner rights then you are able to give the initial "Kick" to the process to make everything start. At this point Service accounts starts to enter in the process and they taking care of creating all the resource for you, automatically.
These service account have different powers and permissions (that can be customised) in order to limit the attack surface in case of one of them is compromise and to keep a sort of order, you will have for example ****-compute#developer.gservuceaccount.com that is a Default compute engine service account.
When you enable different the API some of these service accounts can be created in order to make the components work as expected, but if one of them is deleted or modified you might face one of the error that you are experiencing.
Usually the easiest way to solve the issue is recreate the service account for example deleting it and disabling an enabling the corresponting API.
For example when you enable Kubernetes engine service-****#container-engine-robot-iam-gaservice account is created
In my test project for example I modified them removing the "Kubernetes Engine service Agent" permission and I modified as well the Google APIs service account setting it as a "project viewer" and I am facing permission issues both creating and deleting clusters.
You can navigate through IAM&Amin-->admin to check the status and which service accounts are at the moment authorised in your project.
Here you can find a more deep explanation of some default service accounts.
Here you can find a small guide regarding how to re-enable Kubernetes Engine's default service account:
"If you remove this role binding from the service account, the default service account becomes unbound from the project, which can prevent you from deploying applications and performing other cluster operations."
I trying to create some firewall rules in google compute, everything goes well, but some time later, they are just disappears.
I tried to add rules on default network, and also custom created - in both cases result same.
Tried both: through web UI, and through gcloud tool
If you believe that someone or something is reverting your Firewall changes, you can take multiple approaches to verify that.
inspect Cloud Console Activity logs
same using CLI: gcloud beta logging read "resource.type=gce_firewall_rule"
check GCE Operations section in Cloud Console
check GCE API requests in Cloud Console Logging, using this advanced filter:
resource.type="gce_firewall_rule"
jsonPayload.event_subtype:"compute.firewalls"
I have a Google Cloud VM instance, but I overlooked setting the Storage permission to read-write when creating it.
Now further down the line, I'm looking to experiment with cloud storage, but my instance is read-only.
How can this be changed? I understand it may be possible by relaxing the storage buckets policy, but I'd prefer my instance had write access to all future project buckets.
I presume there is an option in gcloud to change the devstorage parameter?
So, you can't change the option to grant the VM permission, but I did find that you can just run
gcloud auth login
then you can sign in with your management account from the VM and you'll be able to run commands that way.
Unfortunately you can't change the scopes of the VM. You will have to create a new one to change them.
When you create a new one you can reuse the disk of the old VM, if that helps avoid the pain.
You can update the permissions of your VM instance now but only when it is shut down. Check the following documentation:
Changing the service account and access scopes for an instance
If you want to update the API permissions for the kuberntes clusters VM instance then you cannot do that unless you create a new cluster and give the API access to the nodes associated with the clusters.
I believe they have added the option to change it now, without creating another VM.
Once you have stopped the instances, click on the instances you want to change. On the top there is a Edit button, you can click on it and change any the permissions.
Hope the image helps
edit button
Once you have changed the permission to read_write and it still says Access Denied. Go in your instance SSH browser window and enter 'gcloud auth login', follow the steps and hopefully it works!
You need to stop your vm first, then click on edit and change the cloud api access scopes for storage.
You may find more information here: https://ismailyenigul.medium.com/setting-access-scope-of-google-cloud-vm-instances-c8637718f453