NoCredentialProviders: no valid providers in chain. Deprecated. (DNSControl) - amazon-web-services

I am trying to implement dnscontrol to use route 53, but it seems it does not see what I put inside creds.json file when I run any command e.g "dnscontrol get-zones gcloud GCLOUD mydomain.com"
It returns the error:
NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
I did place the credentials inside creds.json in the format below:
{
"r53_main": {
"KeyId": "mywhateverkeyid",
"SecretKey": "mywhateversecretkey"
}
}
My gcloud credentials work fine inside the same creds.json file.
Thanks

If you're trying to authenticate against Google (Cloud DNS), you will need to use a (Google) Service Account JSON key as described here.
It appears you may be trying to use Route 53 credentials as described here.
Example
PROJECT=[[PROJECT]]
BILLING=[[BILLING]]
DNSNAME=[[DNSNAME]]
ACCOUNT=[[ACCOUNT]]
gcloud projects create ${PROJECT}
gcloud beta billing projects link ${PROJECT} --billing-account=${BILLING}
gcloud services enable dns.googleapis.com --project=${PROJECT}
# Create a test DNS Managed Zone
gcloud dns managed-zones create test \
--description=test \
--dns-name=${DNSNAME} \
--project=${PROJECT}
# Create a Service Account (permitted to use DNS) and Key
gcloud iam service-accounts create ${ACCOUNT} --project=${PROJECT}
gcloud projects add-iam-policy-binding ${PROJECT} \
--member=serviceAccount:${ACCOUNT}#${PROJECT}.iam.gserviceaccount.com \
--role=roles/dns.admin
gcloud iam service-accounts keys create ./${ACCOUNT}.json \
--iam-account=${ACCOUNT}#${PROJECT}.iam.gserviceaccount.com \
--project=${PROJECT}
You will need to then combine the key into a creds.json:
echo "{\"gcloud\":$(cat ./${ACCOUNT}.json)}" > ./creds.json
Then:
./dnscontrol check-creds gcloud GCLOUD
${DNSNAME}
./dnscontrol get-zones gcloud GCLOUD ${DNSNAME}
$ORIGIN ${DNSNAME}.
$TTL 300
# 21600 IN NS ns-cloud-b1.googledomains.com.
21600 IN NS ns-cloud-b2.googledomains.com.
21600 IN NS ns-cloud-b3.googledomains.com.
21600 IN NS ns-cloud-b4.googledomains.com.

Related

'stateInfo.state' filter not working for GCP IAM Recommender API

I'm trying to query the GCP IAM recommender API (API documentation here) and fetch role revision recommendations for my project. I'm looking for ACTIVE recommendations only. However, the input filter stateInfo.state filter (listed in the above documentation) is not working for me. It returns the error Invalid Filter. Can someone please let me know what am I doing wrong here? Thanks.
Here's my API query: https://recommender.googleapis.com/v1/projects/my-demo-project/locations/global/recommenders/google.iam.policy.Recommender/recommendations?filter=stateInfo.state:ACTIVE
Please include a minimally reproducible example in questions:
PROJECT=[YOUR-PROJECT-ID]
LOCATION="global"
SERVICES=(
"cloudresourcemanager"
"recommender"
)
for SERVICE in ${SERVICES[#]}
do
gcloud services enable ${SERVICE}.googleapis.com \
--project=${PROJECT}
done
ACCOUNT="tester"
EMAIL=${ACCOUNT}#${PROJECT}.iam.gserviceaccount.com
gcloud iam service-accounts create ${ACCOUNT} \
--project=${PROJECT}
# Minimal role for Recommender for IAM
gcloud projects add-iam-policy-binding ${PROJECT} \
--member=serviceAccount:${EMAIL} \
--role=roles/recommender.iamViewer
gcloud iam service-accounts keys create ${PWD}/${ACCOUNT}.json \
--iam-account=${EMAIL} \
--project=${PROJECT}
# Be careful this overwrites the default gcloud auth account
# Remember to revert this to your e.g. me#gmail.com afterwards
gcloud auth activate-service-account --key-file=${PWD}/${ACCOUNT}.json
TOKEN=$(gcloud auth print-access-token ${EMAIL})
RECOMMENDER="google.iam.policy.Recommender"
PARENT="projects/${PROJECT}/locations/${LOCATION}/recommenders/${RECOMMENDER}"
FILTER="stateInfo.state=ACTIVE"
curl \
--header "Authorization: Bearer ${TOKEN}" \
https://recommender.googleapis.com/v1/${PARENT}/recommendations?filter=${FILTER}
Yields (HTTP 200):
{}

Use Identity Aware Proxy to tunnel to a TPU

Can I use google cloud's identity aware proxy to connect to the gRPC endpoint on a TPU worker? By "TPU worker" I mean that I am creating a TPU with no associated compute instance (using gcloud compute tpus create) and I wish to connect to the gRPC endpoint found by running gcloud compute tpus describe my-tpu:
ipAddress: <XXX>
port: <YYY>
I can easily set up an SSH tunnel to connect to this endpoint from my local machine but I would like to use IAP to create that tunnel instead. I have tried the following:
gcloud compute start-iap-tunnel my-tpu 8470
but I get
- The resource 'projects/.../zones/.../instances/my-tpu' was not found
This makes sense because a TPU is a not a compute instance, and the command gcloud compute start-iap-tunnel expects an instance name.
Is there any way to use IAP to tunnel to an arbitrary internal IP address? Or more generally, is there any other way that I can use IAP to create a tunnel to my TPU worker?
Yes, it can be done using the internal ip address of the TPU Worker, here is an example:
gcloud alpha compute start-iap-tunnel \
10.164.0.2 8470 \
--local-host-port="localhost:$LOCAL_PORT" \
--region $REGION \
--network $SUBNET \
--project $PROJECT
Be aware that Private Google Access must be enabled in the TPU subnet, which can be easily done with the following command:
gcloud compute networks subnets update $SUBNET \
--region=$REGION \
--enable-private-ip-google-access
Just as a reference, here you have an example on how to create a TPU Worker with no external ip address:
gcloud alpha compute tpus tpu-vm create \
--project $PROJECT \
--zone $ZONE \
--internal-ips \
--version tpu-vm-tf-2.6.0 \
--accelerator-type v2-8 \
--network $SUBNET \
$NAME
AUTHENTICATION
To successfully authenticate the endpoint source of the IAP tunnel, you need to add the SSH keys to the project's metadata following these steps:
Check if you already have SSH keys generated in your endpoint:
ls -1 ~/.ssh/*
#=>
/. . ./id_rsa
/. . ./id_rsa.pub
If you don't have any, you can generate them with the command: ssh-keygen -t rsa -f ~/.ssh/id_rsa -C id_rsa.
Add the SSH keys to your project's metadata:
gcloud compute project-info add-metadata \
--metadata ssh-keys="$(gcloud compute project-info describe \
--format="value(commonInstanceMetadata.items.filter(key:ssh-keys).firstof(value))")
$(whoami):$(cat ~/.ssh/id_rsa.pub)"
#=>
Updated [https://www.googleapis.com/compute/v1/projects/$GCP_PROJECT_NAME].
Assign the iap.tunnelResourceAccessor role to the user:
gcloud projects add-iam-policy-binding $GCP_PROJECT_NAME \
--member=user:$USER_ID \
--role=roles/iap.tunnelResourceAccessor

Why do I get "get-credentials requires edit permission" error in gcloud on my terminal, when it succeeds in Cloud Shell?

From my laptop, I am able to execute most gcloud commands, for example creating a cluster and many other commands. I have the Project Owner role.
But when I try to get credentials for a K8s cluster, I get a permission error. But in Cloud Shell, the command succeeds.
The logged-in account is the same in both.
% gcloud container clusters get-credentials my-first-cluster-1 --zone us-central1-c --project my-project
Fetching cluster endpoint and auth data.
ERROR: (gcloud.container.clusters.get-credentials) get-credentials requires edit permission on my-project
$ gcloud config list account --format "value(core.account)"
<MY EMAIL>
But in Cloud Shell, this succeeds!
$ gcloud container clusters get-credentials my-first-cluster-1 --zone us-central1-c --project my-project
Fetching cluster endpoint and auth data.
kubeconfig entry generated for my-first-cluster-1.
$ gcloud config list account --format "value(core.account)"
<MY EMAIL>
The error message is indeed incorrect and not very helpful in this case. This issue occurs when the gcloud config value container/use_client_certificate is set to True but no client certificate has been configured (note that client certificate is a legacy authentication method and is disabled by default for clusters created with GKE 1.12 and higher.). Setting it to False via the following gcloud command solves this issue:
gcloud config set container/use_client_certificate False
This config value is set to False by default in Cloud Shell, which explains the different behavior you experienced.

When creating Google Cloud service accounts do you have to authorize the key after you create it?

When you make a Google Cloud service account using the gcloud command line interface there's a gcloud iam service-accounts keys create to create a key. Looking in the web console, it appears that command creates and registers the key with the account.
Is that sufficient to active the service account for use with they generated JSON key file? Or do you also have to call:
gcloud auth activate-service-account <IAM> --key-file=<JSON file from the keys create command>
The Google docs are a little unclear here as to whether that last step is necessary or not. The console shows no changes to the service account but the command executes successfully if you do make the call.
Creating a key via gcloud iam service-accounts keys create does NOT immediately make it available to use with gcloud commands. You indeed need to activate via gcloud auth activate-service-account.
Use
gcloud auth list
to view your set of credentials. Moreover gcloud uses currently active credentials. You can view your current settings by running
gcloud config list
Also it is possible to use various credentials just by adding extra --account flag to any gcloud command. For example:
gcloud compute zones list --account my_account#gmail.com
where account is previously was obtained via gcloud auth login or gcloud auth activate-service-account and appears in gcloud auth list.
You do not have to use activate-service-account. You can instead use the environment variable 'CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE' to specify your service account json key.
CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE="./service-account.json" \
gcloud deployment-manager deployments \
create $DEPLOYMENT \
--project $PROJECT \
--template resources.jinja \
--properties deployment:$DEPLOYMENT,project:$PROJECT
I found this option here:
https://serverfault.com/questions/848580/how-to-use-google-application-credentials-with-gcloud-on-a-server

How do I entitle serviceAccounts via gcloud command-line for Kubernetes API access?

I'm trying to automate creation of service accounts for use with GKE via the gcloud command-line tool. I've figured out a flow that appears to mirror the process used by the Google Cloud Console, but my users don't see to receive the appropriate access.
Here's the commands I'm executing in order:
# Environment:
# - uname=<username>
# - email=<user's email address>
# - GCLOUD_PROJECT_ID=<project identifier>
# - serviceAccount="${uname}#${GCLOUD_PROJECT_ID}.iam.gserviceaccount.com"
$ gcloud iam service-accounts \
create "${uname}" --display-name "email:${email}" --format json
$ gcloud projects \
add-iam-policy-binding "${GCLOUD_PROJECT_ID}" \
--member "serviceAccount:${serviceAccount}" \
--role=roles/container.developer --format=json
$ gcloud iam service-accounts keys \
create "${GCLOUD_PROJECT_ID}-${uname}.json" \
--iam-account="${serviceAccount}"
When this executes, it creates a new service account and generates a key file locally. I then try to use this key to get credentials for my Kubernetes cluster.
$ gcloud config configurations create devcluster --activate
$ gcloud config set project devnet-166017
$ gcloud config set compute/zone us-central1-b
$ gcloud auth activate-service-account \
--key-file="${GCLOUD_PROJECT_ID}-${uname}.json"
$ gcloud container clusters get-credentials devcluster
ERROR: (gcloud.container.clusters.get-credentials) ResponseError: \
code=403, message=Required "container.clusters.get" permission for \
"projects/${GCLOUD_PROJECT_ID}/zones/us-central1-b/clusters/devcluster".
It appears that for some reason my service account doesn't have one of the permissions it needs to get credentials, but based on what I've read and what I've observed in the Console, I believe this permission should be part of the roles/container.developer role.
Thanks!
I assume by service account, you mean the Service Account for Google Cloud. Here are the IAM roles related to GKE: https://cloud.google.com/container-engine/docs/iam-integration (search for container.).
First create a service account:
gcloud iam service-accounts create --display-name "GKE cluster access" gke-test
Then create a key:
gcloud iam service-accounts keys create key.json --iam-account=gke-test#[PROJECT_ID].iam.gserviceaccount.com
Now you need to assign some roles to this service account, your options are:
roles/container.admin Full management of Container Clusters and their Kubernetes API objects.
roles/container.clusterAdmin Management of Container Clusters.
roles/container.developer Full access to Kubernetes API objects inside Container Clusters.
roles/container.viewer Read-only access to Container Engine resources.
Again look at https://cloud.google.com/container-engine/docs/iam-integration page for details.
I assign roles/container.viewer (a read-only role, minimum you can assign to get-credentials) to this service account:
gcloud projects add-iam-policy-binding [PROJECT_ID] --role=roles/container.viewer --member=serviceAccount:gke-test#[PROJECT_ID].iam.gserviceaccount.com
Logout on gcloud from your current account:
gcloud auth revoke
Login to gcloud with the service account key:
gcloud auth activate-service-account --key-file=key.json
Try get-credentials:
$ gcloud container clusters get-credentials test --zone us-west1-a
Fetching cluster endpoint and auth data.
kubeconfig entry generated for test.
It works. I tried it with roles/container.developer, which also works.
You can try other permissions and see what works and what doesn't, although you made it clear that the documentation doesn't make it clear which roles have access to container.clusters.getCredentials.