gcloud compute no longer available after migration to v2beta2 - google-cloud-platform

After running: gcloud components update beta
It seems I lost compute commands:
gcloud -h
Usage: gcloud [optional flags] <group | command>
group may be auth | beta | components | config | dns | preview |
topic
command may be docker | help | info | init | version
How do I get compute back in order to run simple things like: gcloud compute images list --project google-containers | grep container-vm?
I followed migration path from: https://cloud.google.com/deployment-manager/latest/migration-guide
This is my gcloud -v:
Google Cloud SDK 0.9.67
beta 2015.06.30
bq 2.0.18
bq-nix 2.0.18
core 2015.06.30
core-nix 2015.06.02
gcloud 2015.06.30
gcutil-msg 2015.06.09
gsutil 4.13
gsutil-nix 4.12
preview 2015.06.30
Everytime I run the compute command, console gets in loop until I kill with ctrl+c:
[gcloud _19:33:01 $]]^_^ gcloud compute -h
You do not currently have this command group installed. Using it
requires the installation of components: [compute]
WARNING: Component [compute] no longer exists.
All components are up to date.
Restarting gcloud command:
$ gcloud compute -h
You do not currently have this command group installed. Using it
requires the installation of components: [compute]
WARNING: Component [compute] no longer exists.
All components are up to date.
Restarting gcloud command:
$ gcloud compute -h
^C
Is there anything I missed?
cheers
Leo

I had same error after updating my gcloud tools. I had to replace the current one by re-installing the gcloud kit to make it work.

Related

how to run airflow cli command for composer?

This command works as it should:
gcloud composer environments run <COMPOSER_NAME> dags next-execution -- <DAG_ID>
But when I add any option to airflow command eg: -n 5:
gcloud composer environments run <COMPOSER_NAME> dags next-execution -- -n 5 <DAG_ID>
It throws error:
kubeconfig entry generated for europe-west1-<ID>.
Executing within the following Kubernetes cluster namespace: composer-<ID2>
Error from server (NotFound): pods "airflow-worker-<ID3>" not found
ERROR: (gcloud.composer.environments.run) kubectl returned non-zero status code.
How do I pass -n 5 to airflow command properly?
Thank you
With gcloud beta, you should be able to use Airflow sub commands, example :
gcloud beta composer environments run myenv trigger_dag -- some_dag --run_id=foo
is equivalent to running the following command from a shell inside the my-environment environment:
airflow trigger_dag some_dag --run_id=foo
I don't tested the command but it should look like :
gcloud beta composer environments run <COMPOSER_NAME> next-execution -- -n 5 -- dag_id
airflow next_execution [-h] [-sd SUBDIR] dag_id
Before to launch all your commands, update the gcloud cli from you Shell session :
gcloud components update

What does `gcloud compute instances create` do? - POST https://compute.googleapis.com…

Some things are very easy to do with the gcloud CLI, like:
$ export network='default' instance='example-instance' firewall='ssh-http-icmp-fw'
$ gcloud compute networks create "$network"
$ gcloud compute firewall-rules create "$firewall" --network "$network" \
--allow 'tcp:22,tcp:80,icmp'
$ gcloud compute instances create "$instance" --network "$network" \
--tags 'http-server' \
--metadata \
startup-script='#! /bin/bash
# Installs apache and a custom homepage
apt update
apt -y install apache2
cat <<EOF > /var/www/html/index.html
<html><body><h1>Hello World</h1>
<p>This page was created from a start up script.</p>
</body></html>'
$ # sleep 15s
$ curl $(gcloud compute instances list --filter='name=('"$instance"')' \
--format='value(EXTERNAL_IP)')
(to be exhaustive in commands, tear down with)
$ gcloud compute networks delete -q "$network"
$ gcloud compute firewall-rules delete -q "$firewall"
$ gcloud compute instances delete -q "$instance"
…but it's not clear what the equivalent commands are from the REST API side. Especially considering the HUGE number of options, e.g., at https://cloud.google.com/compute/docs/reference/rest/v1/instances/insert
So I was thinking to just steal whatever gcloud does internally when I write my custom REST API client for Google Cloud's Compute Engine.
Running rg I found a bunch of these lines:
https://github.com/googleapis/google-auth-library-python/blob/b1a12d2/google/auth/transport/requests.py#L182
Specifically these 5 in lib/third_party:
google/auth/transport/{_aiohttp_requests.py,requests.py,_http_client.py,urllib3.py}
google_auth_httplib2/__init__.py
Below each of them I added _LOGGER.debug("With body: %s", body). But there seems to be some fancy batching going on because I almost never get that With body line 😞
Now messing with Wireshark to see what I can find, but I'm confident this is a bad rabbit hole to fall down. Ditto for https://console.cloud.google.com/home/activity.
How can I find out what body is being set by gcloud?
Add the command line option --log-http to see the REST API parameters.
There is no simple answer as the CLI changes over time. New features are added, removed, etc.

ERROR: gcloud crashed (TypeError): setproxy() takes at most 7 arguments (8 given)

Both gcloud auth login and gcloud compute --project XXX ssh --zone YYY ZZZ printed the error message in the title.
ERROR: gcloud crashed (TypeError): setproxy() takes at most 7 arguments (8 given)
The gcloud version is
Google Cloud SDK 186.0.0
bq 2.0.28
core 2018.01.22
gsutil 4.28
gcloud auth login has worked before the upgrade to 186.0.0 (from 182.0.0). All of gcloud has worked some time previously. To double-check, the proxy has been set:
gcloud config set proxy/address 127.0.0.1
gcloud config set proxy/port 8080
gcloud config set proxy/type http
but this has not changed anything (proxy is set via environment variables anyways).
https_proxy=http://127.0.0.1:8080
http_proxy=http://127.0.0.1:8080
UPDATE: The error has persisted even after removing the config directory ~/.config/gcloud. There's an issue at https://issuetracker.google.com/issues/72407295.
How can this be fixed?
As a workaround, downgrading to 185 solved the issue:
Go to https://storage.cloud.google.com/cloud-sdk-release, type "google-cloud-sdk-185" into the search bar, download the file for your OS, unzip, etc. You need to use a new shell to unset the change to the PATH that gcloud did.
It has been fixed in 187:
We've found a fix; expect it in the 187.0.0 Cloud SDK release next week.
Update Version 187 has been released: Download at https://cloud.google.com/sdk/downloads#versioned. This also gives the message
To revert your SDK to the previously installed version, you may run:
$ gcloud components update --version 185.0.0
after upgrade, so that could be used to revert as well, if similar problems occur.

Google Codelab - Image Classification Transfer Learning - gcloud beta ml init-project error

I'm starting the Google Codelab on transfer learning with GCP: After installing the cloud sdk in the Cloud shell:
sudo pip install --upgrade pillow
curl https://storage.googleapis.com/cloud-ml/scripts/setup_cloud_shell.sh | bash
export PATH=${HOME}/.local/bin:${PATH}
I cannot go beyond the following command:
gcloud beta ml init-project
which returns the following:
ERROR: (gcloud.beta.ml) Invalid choice: 'init-project'.
Usage: gcloud beta ml [optional flags] <group>
group may be language | speech | video | vision
For detailed information on this command and its flags, run:
gcloud beta ml --help
Ref: https://codelabs.developers.google.com/codelabs/cpb102-txf-learning/index.html?index=..%2F..%2Findex#2
I guess the codelab is old and the SDK has been changed. Can anybody point me to documentation that is perhaps more up-to-date for using the cloud sdk to do transfer learning?
Thanks!
Here is a more up to date codelabs that steps you through transfer learning:
https://codelabs.developers.google.com/codelabs/scd-coastline/index.html
That init-project call is no longer required; it happens automatically.

Get the Default GCP Project ID with a Cloud SDK CLI One-Liner

I’m looking for a gcloud one-liner to get the default project ID ($GCP_PROJECT_ID).
The list command gives me:
gcloud config list core/project
#=>
[core]
project = $GCP_PROJECT_ID
Your active configuration is: [default]
While I only want the following output:
gcloud . . .
#=>
$GCP_PROJECT_ID
The easiest way to do this is to use the --format flag with gcloud:
gcloud config list --format 'value(core.project)' 2>/dev/null
The --format flag is available on all commands and gives you full control over what is printed, and how it is formatted.
You can see this help page for full info:
gcloud topic formats
Thanks to comment from Tim Swast above, I was able to use:
export PROJECT_ID=$(gcloud config get-value project)
to get the project ID. Running the get-value command prints the following:
gcloud config get-value project
#=>
Your active configuration is: [default]
$PROJECT_ID
You can also run:
gcloud config get-value project 2> /dev/null
to just print $PROJECT_ID and suppress other warnings/errors.
With Google Cloud SDK 266.0.0 you can use following command:
gcloud config get-value project
Not exactly the gcloud command you specified, but will return you the currently configured project:
gcloud info |tr -d '[]' | awk '/project:/ {print $2}'
Works for account, zone and region as well.
From Cloud Shell or any machine where Cloud SDK is installed, we can use:
echo $DEVSHELL_PROJECT_ID
And as shown in the below screenshot.
I got a question about how to set the environment variable $DEVSHELL_PROJECT_ID; here are the steps:
If the URL has the variable project and is set to some project id, then the environment variable $DEVSHELL_PROJECT_ID usually will be set to the project id.
If the variable project is not set in the URL, we can choose the project from the Combobox (besides the title Google Cloud Platform) which will set the variable project in the URL. We may need to restart the Cloud Shell or refresh the entire web page to set the environment variable $DEVSHELL_PROJECT_ID.
Otherwise, if the environment variable $DEVSHELL_PROJECT_ID is not set, we can set it by the command shown below where we replace PROJECT_ID with the actual project id.
gcloud config set project PROJECT_ID
All these are shown in the below figure.
Direct and easy way to get the default $PROJECT_ID is answered above.
In case you would like to get $PROJECT_ID from the info command, here is a way to do it:
gcloud info --format=flattened | awk '/config.project/ {print $2}'
or:
gcloud info --format=json | jq '.config.project' | tr -d '"'
Just run:
gcloud info --format={flattened|json}
to see the output, then use awk, jq or similar tools to grab what you need.