authentication for GCP STT Quickstart problem - google-cloud-platform

I am following the GCP Speech-to-Text Quickstart. As best as I can tell, I have followed all setup criteria.
Enabled STT for my project.
Generated Service API keys and downloaded the JSON file.
Installed SDK and initialized.
In Windows CMD shell, I set GOOGLE_APPLICATION_CREDENTIALS to the downloaded JSON file.
In Windows CMD shell, I executed gcloud auth activate-service-account <my service email generated by GCP> --key-file= "mypath/JSON key file".
I executed gcloud auth list and I see my project account identifier.
I executed the example curl command:
curl -s -H "Content-Type: application/json" -H "Authorization: Bearer "$(gcloud auth application-default print-access-token) https://speech.googleapis.com/v1/speech:recognize -d #sync-request.json
And get this error:
{
"error": {
"code": 401,
"message": "Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
"status": "UNAUTHENTICATED"
}
}
No where in the Quickstart steps does it mention OAuth
As a test, I executed:
gcloud auth application-default print-access-token
And got this:
(gcloud.auth.application-default.print-access-token) File "mypath/JSON key file" was not found.
Even though the file exists in the folder I specify.
Trying something else, I tried executing the Java example in the SDK. It creates a very simple SpeechClient with no credentials, which seems suspect. I made the GOOGLE_APPLICATION_CREDENTIALS env variable available to the example. I think the example uses gRCP, but not sure.
The example hangs at:
RecognizeResponse response = speech.recognize(config, audio);
Looking online, I found the likely suspect is bad authentication, which is the same as trying the CMD line example.
Any and all guidance is appreciated.

Did you run the curl command from the same directory where your JSON key file is located?
Google's documentation states the following:
Note that to pass a filename to curl you use the -d option (for
"data") and precede the filename with an # sign. This file should be
in the same directory in which you execute the curl command.

I have the answer to the CLI issue. A dumb mistake on my part. When I set GOOGLE_APPLICATION_CREDENTIALS I wrapped the pathname in double quotes. Sigh. I reset the env variable without the double quotes.
I could successfully run gcloud auth application-default print-access-token and it printed out the token.
I tried the curl command again with $(gcloud auth....) and got same error. Next, I tried the curl command replacing the $(gcloud auth....) with the token returned above and it worked!
Next, I need to resolve the Java example and I am good.

No need to be suspicious:
If you don't specify credentials when constructing the client, the client library will look for credentials via the environment variable GOOGLE_APPLICATION_CREDENTIALS.
In your java code try to print System.getenv("GOOGLE_APPLICATION_CREDENTIALS"), to verify it's set . Probably it's not, depending on how you are setting it in your IDE, or terminal.

Related

How do I programmatically download a file from a private Google Cloud Source Repository with a service account?

I have a Google Cloud Source Repository I want my application to download files from. I have a specific use case where I want to get files from a Google Cloud Source Repository programmatically- not GCS or another location.
I want to control permissions to the repo with standard Google IAM. Can I grant a GCP service account access to read from a Cloud Source Repository?
In bitbucket you can download a file directly from a private repo with a rest call like this: curl -s -S --user username:apppassword -L -O https://bitbucket.org/<ORG_NAME>/<REPO>/src/master/<FOLDER>/file.txt
How can I use a GSA to download a file like this from a private Google Cloud Source Repository?
I am doing this in code so I do not have access to ssh or curl or the gcloud cli. I'll be using python to fetch this file.
I was also looking if the SDK supports this. I did not see anything in the docs for a python API for interacting with Google Cloud Source Repositories this way. I'm wondering how I can pull down this file with the requests library or even something like GitPython while authenticating with the GSA.
EDIT
Per the comments I tried creating a token in python and gcloud, but it does not work. The token is generated fine, but file download doesn't work.
I tried this (and via python):
curl -s -S -H "Authorization: Bearer $(gcloud auth print-access-token)" -L -O https://source.cloud.google.com/MY_GCP_PROJECT/MY_REPO/master/README.md
This downloads a huge html page that seems to be showing auth errors.
Maybe the http path is wrong? What is the correct path to the file in the source repo via http GET?
I confirmed I have permissions because this works gcloud source repos clone MY_REPO --project=MY_PROJECT
EDIT
This is where I am right now, I can't figure out what the right URL is to point to a specific branch and file:
import google.auth
import google.auth.transport.requests
import requests
# Generate a token from current security context
creds, project = google.auth.default()
auth_req = google.auth.transport.requests.Request()
creds.refresh(auth_req)
# Set token in Authorization header of http request
headers = {'Authorization':'Bearer {}'.format(creds.token)}
# Repo URL with branch and file specified (trying to download README.md in the root of the repo)
# What is the right URL here?
url = "https://source.developers.google.com/p/<GCP PROJECT>/r/<REPO NAME>/<BRANCH NAME>/README.md"
response = requests.get(url, headers=headers)
# I get a big mess of html with auth errors
print(response.content)
If I use this URL "https://source.developers.google.com/<GCP PROJECT>/<REPO NAME>/<BRANCH NAME>/README.md" I get back a page that includes PERMISSION_DENIED: The caller does not have permission

ERROR: gcloud crashed (ServerNotFoundError): Unable to find the server at www.googleapis.com

I am trying to sign in to the cloud sdk with the command: gcloud auth login, and I select my google account in the browser. After I click allow, in the terminal it says:
ERROR: gcloud crashed (ServerNotFoundError): Unable to find the server at www.googleapis.com
If you would like to report this issue, please run the following command:
gcloud feedback
To check gcloud for common problems, please run the following command:
gcloud info --run-diagnostics
And when I run the command gcloud info --run-diagnostics it also stops with the error:
ERROR: Reachability Check failed.
Cannot reach https://www.googleapis.com/auth/cloud-platform (ServerNotFoundError)
Network connection problems may be due to proxy or firewall settings.
My config is the default one without any modifications.
I could sign in with no issues to the cloud sdk for a long time.
I am on windows 10.
I tried signing in both with the cloud sdk shell and the windows terminal, as administrators and not as administrators.
How do I fix this error?
UPDATE:
I run the tracert -4 www.googleapis.com and also -6 command and this is the result:
Unable to resolve target system name www.googleapis.com.
I am working from home, and I don't know what a network proxy is, I might be accidentally using one.
You may have enabled proxy with gcloud, use-> gcloud config list to get the proxy settings
To unset proxy use: gcloud config unset proxy/[params] where params are address, port etc.
You need to login into your gcloud SDK first using this command
gcloud auth login
It will open a google sign up page in the browser. Select your account and then you will get a conformation in you command line that you have been authenticated. Then try what you wanted to do.
I faced the same issue when connected to VPN. Disconnected from VPN and ran the below command and it worked.
gcloud auth login

Google Cloud - Wrong project id being used from different email address

Despite running gcloud auth application-default login and gcloud config set core/project CORRECT_PROJECT_ID the project keeps defaulting to an incorrect project id:
gcloud config list
[core]
account = CORRECT_EMAIL
disable_usage_reporting = True
project = CORRECT_PROJECT_ID
Your active configuration is: [default]
I can successfully run the sample code from the tutorial (below) if I run in the terminal
export GOOGLE_APPLICATION_CREDENTIALS="[PATH]"
However, I didn't want to have to do this every time, so I ran the command:
gcloud auth application-default login
This opened a browser with a list of my gmail accounts, and even though I selected the correct account, the success window went to a different gmail account. So then I tried it in an incognito window, and it worked.
However, running npm start resulted in the following error:
ERROR: { Error: 7 PERMISSION_DENIED: Cloud Natural Language API has not been used in project WRONG_PROJECT_ID before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/language.googleapis.com/overview?project=WRONG_PROJECT_ID then retry.
Then I ran gcloud config set core/project CORRECT_PROJECT_ID and got the message Updated property [core/project].
When I run npm start I get the same message:
ERROR: { Error: 7 PERMISSION_DENIED: Cloud Natural Language API has not been used in project WRONG_PROJECT_ID before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/language.googleapis.com/overview?project=WRONG_PROJECT_ID then retry.
I tried gcloud auth login and got the following message (after I authenticated in an incognito window):
WARNING: `gcloud auth login` no longer writes application default credentials.
If you need to use ADC, see:
gcloud auth application-default --help
You are now logged in as [CORRECT EMAIL ADDRESS].
Your current project is [CORRECT_PROJECT_ID]. You can change this setting by running:
$ gcloud config set project PROJECT_ID
I have a few suggestions that may correct this behaviour.
1) Clear your web browser cache & cookies. Then run "gcloud auth application-default login"
2) Try re-installing the gcloud toolkit.
3) Try unsetting the project in your config first, then set the project to the correct project. i.e.
gcloud config unset project WRONG_PROJECT_ID
gcloud config set project CORRECT_PROJECT_ID
4) Check the “CLOUDSDK_CORE_PROJECT” environment variable. Set it to the correct project if it is not already.
5) Try re-running “gcloud init”
6) You can find your application default credentials in
Linux: ~/.config/gcloud/application_default_credentials.json
Windows: C:\Users\%username%\AppData\Roaming\gcloud\credentials
You can delete the file, & regenerate it using commands you had mentioned in your question such as “gcloud auth default-credentials login”
We need to find where npm start is getting its credentials from. Once we figure that out, we can figure out how to change it, & understand why it’s looking there etc.
Are you able to find the config file to see where it is looking for credentials?
Is npm start the entire command? I’m not too familiar with Node JS. I’m not sure why it is trying to use Natural Language API.
You also mentioned a tutorial but I think you may have forgotten to include it in your question. Which tutorial are you referring to?

Invalid audio source error in google cloud speech API

I have followed google's tutorial with gcloud tool to set up everything to use the cloud speech API. However when I am trying to send the following request:
gcloud ml speech recognize 'gs://cloud-samples-tests/speech/brooklyn.flac' --language-code='en-US'
I keep getting the following error:
ERROR: (gcloud.ml.speech.recognize) Invalid audio source ['gs://cloud-samples-tests/speech/brooklyn.flac']. The source must either be a local path or a Google Cloud Storage URL (such as gs://bucket/object).
I also tried google's tutorial to use the speech API from command file using curl request... but when I sent the following request I haven't got any response
curl -s -H "Content-Type: application/json"
-H "Authorization: Bearer "$(gcloud auth print-access-token)
https://speech.googleapis.com/v1/speech:recognize
-d #sync-request.json
I don't know what I am doing wrong... Any help would be really appreciated.. Thanks in advance
The commenter is exactly right, for some reason the quotes to the file argument are the problem. This appears to be true for both local files and Google Cloud Storage hosted file. I had the exact same problem and removing the quotes cures things. It's possible that this is a platform specific issue - I am using gcloud on Windows 10.
I had similar issue. Finally figured out that I had to remove the backslash after the audio file name brooklyn.flac
Gcloud Quickstart has it like this:
gcloud ml speech recognize gs://cloud-samples-tests/speech/brooklyn.flac \ --language-code=en-US
I just used the below after removing the backslash:
gcloud ml speech recognize gs://cloud-samples-tests/speech/brooklyn.flac --language-code=en-US
For me, on the command line of windows 7, the following finally worked:
gcloud ml speech recognize gs://cloud-samples-tests/speech/brooklyn.flac --language-code="en-US"
I had this same issue on Mac OS when referencing a local file. When I deleted the quotes, it worked fine.
This did not work
gcloud ml speech recognize-long-running '/Users/interview/STEREO/FOLDER01/ZOOM0001.WAV'
--language-code='en-US' --async
Deleting the quotes like below did. Go figure.
gcloud ml speech recognize-long-running /Users/interview/STEREO/FOLDER01/ZOOM0001.WAV
--language-code='en-US' --async

request.META does not contain header passed from curl -H

"It works on my machine."
I have a django app. I'm followed this tutorial. OAuth2 works great on my dev box like this:
$ curl -v -H "Authorization: OAuth c52676b24a63b79a564b4ed38db3ac5439e51d47" http://localhost:8000/api/v1/my-model/?format=json
My local dev app finds the header with this line of code:
auth_header_value = request.META.get('HTTP_AUTHORIZATION')
But when I deploy it to my ubuntu box running apache it doesn't.
I added the following to my authentication.py file so I could inspect the values in the log on the remote machine.
logging.error(request.GET)
logging.error(request.POST)
logging.error(request.META)
The header value is mysteriously missing from the output. So I just get 401s.
Did you turn on WSGIPassAuthorization?
http://modwsgi.readthedocs.org/en/latest/configuration-directives/WSGIPassAuthorization.html
Authorisation headers are not passed through by default as doing so
could leak information about passwords through to a WSGI application
which should not be able to see them when Apache is performing
authorisation.