I just created a model on ML Engine with:
gcloud ml-engine models create test_model --enable-logging
I went into the GUI and created a version. I'm hitting this model for predictions but where do I go in the GUI to find the logs for online predictions?
Thanks!
The logs can be found in StackDriver Logging:
Go to https://console.cloud.google.com
Click on the "hamburger" icon in the top left
Find the "Logging" option under "STACKDRIVER"
Click on "Logs" (you can get directly here with a link similar to: https://console.cloud.google.com/logs/viewer?project=my_project, just subsituate your actual project name)
Locate the drop down menu that allows you to select your logs.
Hover over "Cloud ML Model Version
Either click the model you're interested in or hover over it, if you want to select a specific version
(Optional) If selecting a specific version, click on it.
That said, I'll file a feature request to have a link in a more convenient place alongside the model and/or version.
Related
I need to list out all the instance, container, function, notebooks, bucket, dataproc and composer running under project in all the region/locations.
Is it possible to list resources of all the regions location. Gcloud or python script both can work for me
My ultimate goal after listing is to put tag as per its name of the resource.
Thanks
You can use Google Asset inventory feature and query your project like this
gcloud asset search-all-resources --scope=projects/<PROJECT_ID> --page-size=500 --format=json
More detail in the documentation about the query format.
All the ressources aren't supported. You can find the full list here (For example, Cloud Run isn't yet supported, but it's coming soon!)
If you want to access through console, you could go to IAM & Admin Menu, then select Asset Inventory.
Then you could see bunch of asset list.
Click Resource tab if you want download all the details in csv format.
In search asset you will get abundance of irrelevant data. Better to use resource api of the resource you think relevant to you. Like
compute.googleapis.com/Instance
storage.googleapis.com/Bucket
dataproc.googleapis.com/Cluster
container.googleapis.com/Cluster
cloudfunctions.googleapis.com/CloudFunction
dataflow.googleapis.com/Job //Notebook
gcloud asset search-all-resources --asset-types='compute.googleapis.com/Instance,storage.googleapis.com/Bucket' --query='labels.name:*' --format='table(name, assetType, labels)'”
How can I see the logs regarding the VMs that has been created on Google Cloud Compute Engine ?
Google Operations Logging (formerly Stackdriver) provides this information.
To see the details on Google Compute Engine instances that were created in a project, filter based upon the API operation v1.compute.instances.insert and the resouce type resource.type=gce_instance.
Note: the resouce type is not always necessary but it is best to be declarative in what logging should search for.
Example using the CLI:
gcloud logging read "resource.type=gce_instance protoPayload.methodName=v1.compute.instances.insert" --format json
Example using the Google Cloud Console - Legacy Logs Viewer
Go to https://console.cloud.google.com/logs/viewer
Verify that the desired Google Cloud project is displayed in the title area.
In the box showing the text "Filter by label or text search" click the dropdown and select "Convert to advanced filter".
In the box showing the text "Filter by label or text search" enter the following (as two lines in the filter box):
resource.type="gce_instance"
protoPayload.methodName="v1.compute.instances.insert"
Select the desired date search range in the box "Last hour"
Click the "Submit Filter" button
A great feature in debugging Google Cloud Functions calls is the highlighting of the functionName and the execution id (see photo). Is it possible to also get your own logs (generated by the python logging client) to show up highlighted?
You can use summary fields to show these values
https://cloud.google.com/logging/docs/view/logs-viewer-interface#add_summary_fields
Maybe I missed something, under https://console.cloud.google.com/apis/api/sqladmin.googleapis.com/overview
I saw there are a lot of errors, but when I go to Logs Viewer, I couldn't find anything. Is any way I can obtain the error log?
Basically, you should create a query to obtain the data you need in the Log Viewer UI: specify a type of resource and an instance name whose logs you want to view.
GCP Console => Operations => Logging => Logs Viewer
=> Query builder => Resource
Cloud SQL Database = my-project:my-sql-instance
The query builder will show a query preview like below:
resource.type="cloudsql_database"
resource.labels.database_id="my-project:my-sql-instance"
Once you click the “Run Query” button, the log entries will appear. By default log entries for the last 1 hour are shown. You can use the "Edit time" option to change this.
Please see Cloud Logging > Doc > Basic logs queries for more details.
I find myself toggling between different 'resources' to view my logs in stackdriver. For example:
Is there a way to view all combined logs from a single view? Most go to "Global", but many do not as well.
Just convert the filter to advanced view and select the resource types that you need
resource.type="bigquery_resource" OR resource.type="gce_instance" OR resource.type="audited_resource"
That will show you the logs for bigquery and audit and gce in a single view.
I believe you can do textual search across all logs, but it will not be as granular as resource type based search. You can try this with the Advanced filter feature.
On the top right corner of the search box, click the drop down menu and select "Convert to advanced filter". Then delete the "resource.type=xyz" line and type the string you want to search.
As alternative the gcloud sdk cli to search the logs without specifying resource type - for instance, gcloud logging read error --limit 1 would look for "error" string across all your logs and show only 1 matching entry.