Request Google cloud speech-to-text API gives 503 error? - google-cloud-platform

I ran the following python sample code on this page (you need a private key to run this): https://cloud.google.com/speech-to-text/docs/quickstart-client-libraries
# Imports the Google Cloud client library
from google.cloud import speech
import os
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'key.json'
# Instantiates a client
client = speech.SpeechClient()
# The name of the audio file to transcribe
gcs_uri = "gs://cloud-samples-data/speech/brooklyn_bridge.raw"
audio = speech.RecognitionAudio(uri=gcs_uri)
config = speech.RecognitionConfig(
encoding=speech.RecognitionConfig.AudioEncoding.LINEAR16,
sample_rate_hertz=16000,
language_code="en-US",
)
# Detects speech in the audio file
response = client.recognize(config=config, audio=audio)
for result in response.results:
print("Transcript: {}".format(result.alternatives[0].transcript))
This results in the following error:
Traceback (most recent call last):
File "C:\Users\98274\anaconda3\envs\carla\lib\site-packages\google\api_core\grpc_helpers.py", line 57, in error_remapped_callable
return callable_(*args, **kwargs)
File "C:\Users\98274\anaconda3\envs\carla\lib\site-packages\grpc\_channel.py", line 826, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "C:\Users\98274\anaconda3\envs\carla\lib\site-packages\grpc\_channel.py", line 729, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses"
debug_error_string = "{"created":"#1614347442.192000000","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":4090,"referenced_errors":[{"created":"#1614347442.192000000","description":"failed to connect to all addresses","file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc","file_line":394,"grpc_status":14}]}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\98274\anaconda3\envs\carla\lib\site-packages\google\api_core\retry.py", line 184, in retry_target
return target()
File "C:\Users\98274\anaconda3\envs\carla\lib\site-packages\google\api_core\grpc_helpers.py", line 59, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "<string>", line 3, in raise_from
google.api_core.exceptions.ServiceUnavailable: 503 failed to connect to all addresses
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:/Users/98274/Desktop/carla_ppo/test_connection.py", line 22, in <module>
response = client.recognize(config=config, audio=audio)
File "C:\Users\98274\anaconda3\envs\carla\lib\site-packages\google\cloud\speech_v1\services\speech\client.py", line 334, in recognize
response = rpc(request, retry=retry, timeout=timeout, metadata=metadata,)
File "C:\Users\98274\anaconda3\envs\carla\lib\site-packages\google\api_core\gapic_v1\method.py", line 145, in __call__
return wrapped_func(*args, **kwargs)
File "C:\Users\98274\anaconda3\envs\carla\lib\site-packages\google\api_core\retry.py", line 286, in retry_wrapped_func
on_error=on_error,
File "C:\Users\98274\anaconda3\envs\carla\lib\site-packages\google\api_core\retry.py", line 206, in retry_target
last_exc,
File "<string>", line 3, in raise_from
google.api_core.exceptions.RetryError: Deadline of 120.0s exceeded while calling functools.partial(<function _wrap_unary_errors.<locals>.error_remapped_callable at 0x0000026B56606A68>, config {
encoding: LINEAR16
sample_rate_hertz: 16000
language_code: "en-US"
}
audio {
uri: "gs://cloud-samples-data/speech/brooklyn_bridge.raw"
}
, metadata=[('x-goog-api-client', 'gl-python/3.7.9 grpc/1.31.0 gax/1.23.0 gapic/2.0.1')]), last exception: 503 failed to connect to all addresses
What's the problem? Does it related to internet issue because I am in China? How can I solve it?

I am posting the answer as CommunityWiki to further contribuite to the community.
As discussed in the comment section, the code provided in the documentation is running smoothly. However, you have to make sure that you set your credentials using GOOGLE_APPLICATION_CREDENTIALS so your request is authenticated. Also,the client libraries have to be properly installed.
In #antaressgzz case's the request to the API was being blocked by a firewall rule. Thus, it is a good practice to check these rules, changing them if applicable.

Related

GCP Removed Access To Google Collab User

I used to be able to use secretmanager to get secrets from my GCP account using google collab.
Now, whenever I try to run the following code:
client = secretmanager.SecretManagerServiceClient()
name = f"projects/my_project_here/secrets/my_secret_name_here/versions/latest"
response = client.access_secret_version(request={"name": name})
I get the following error over and over:
ERROR:grpc._plugin_wrapping:AuthMetadataPluginCallback "<google.auth.transport.grpc.AuthMetadataPlugin object at 0x7fb313b41850>" raised exception!
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/google/auth/compute_engine/credentials.py", line 111, in refresh
self._retrieve_info(request)
File "/usr/local/lib/python3.7/dist-packages/google/auth/compute_engine/credentials.py", line 88, in _retrieve_info
request, service_account=self._service_account_email
File "/usr/local/lib/python3.7/dist-packages/google/auth/compute_engine/_metadata.py", line 234, in get_service_account_info
return get(request, path, params={"recursive": "true"})
File "/usr/local/lib/python3.7/dist-packages/google/auth/compute_engine/_metadata.py", line 187, in get
response,
google.auth.exceptions.TransportError: ("Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true from the Google Compute Enginemetadata service. Status: 404 Response:\nb''", <google.auth.transport.requests._Response object at 0x7fb313b9c290>)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/grpc/_plugin_wrapping.py", line 90, in __call__
context, _AuthMetadataPluginCallback(callback_state, callback))
File "/usr/local/lib/python3.7/dist-packages/google/auth/transport/grpc.py", line 101, in __call__
callback(self._get_authorization_headers(context), None)
File "/usr/local/lib/python3.7/dist-packages/google/auth/transport/grpc.py", line 88, in _get_authorization_headers
self._request, context.method_name, context.service_url, headers
File "/usr/local/lib/python3.7/dist-packages/google/auth/credentials.py", line 133, in before_request
self.refresh(request)
File "/usr/local/lib/python3.7/dist-packages/google/auth/compute_engine/credentials.py", line 117, in refresh
six.raise_from(new_exc, caught_exc)
File "<string>", line 3, in raise_from
google.auth.exceptions.RefreshError: ("Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true from the Google Compute Enginemetadata service. Status: 404 Response:\nb''", <google.auth.transport.requests._Response object at 0x7fb313b9c290>)
How can I see what user I am in google collab and then add this user in GCP so I can again fetch these secrets?
I know to go to GCP->Secret Manager->My_Secret->Permissions->+Grant Access, but I don't know how to know 1) Who to Add 2) Why this permission changed on its own with no intervention on anyone's end.
Both were originally running under my email (and they are still) so this worked without me ever having to touch secrets access because both were the App Engine default service account<->Secret Manager Secret Accessor

exception: 503 failed to connect to all addresses when pulishing data to Google Pub/Sub

I was using the sample code in the Google Pub/Sub tutorial. When tried to publish the messages, exception: 503 failed to connect to all addresses was thrown. I gave Pub/Sub publisher role to the service account. Everything was working fine until day-before-yesterday.
There was no issue when publishing a message from Console or gcloud commands. Cloud Functions is my subscription.
Here's the link for tutorial:
https://cloud.google.com/pubsub/docs/samples/pubsub-publish-with-error-handler
Following are the packages and their versions:
cachetools==4.2.2 certifi==2021.5.30 charset-normalizer==2.0.3
google-api-core==1.31.0
google-auth==1.33.1
google-cloud-pubsub==2.6.1
googleapis-common-protos==1.53.0
grpc-google-iam-v1==0.12.3
grpcio==1.39.0
idna==3.2 libcst==0.3.19 mypy-extensions==0.4.3 packaging==21.0 proto-plus==1.19.0
protobuf==3.17.3 pyasn1==0.4.8 pyasn1-modules==0.2.8 pyparsing==2.4.7 pytz==2021.1
PyYAML==5.4.1 requests==2.26.0 rsa==4.7.2 six==1.16.0 typing-extensions==3.10.0.0
typing-inspect==0.7.1 urllib3==1.26.6
Here's the error code:
$ python publish-message.py
Failed to publish 10 messages.
Traceback (most recent call last):
File "E:\gcp-pubsub-cs\lib\site-packages\google\api_core\grpc_helpers.py", line 67, in error_remapped_callable
return callable_(*args, **kwargs)
File "E:\gcp-pubsub-cs\lib\site-packages\grpc\_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "E:\gcp-pubsub-cs\lib\site-packages\grpc\_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses"
debug_error_string = "{"created":"#1626984490.444000000","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3009,"referenced_errors":[{"created":"#1626984490.444000000","description":"failed to connect to all addresses","file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc","file_line":398,"grpc_status":14}]}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "E:\gcp-pubsub-cs\lib\site-packages\google\api_core\retry.py", line 188, in retry_target
return target()
File "E:\gcp-pubsub-cs\lib\site-packages\google\api_core\timeout.py", line 102, in func_with_timeout
return func(*args, **kwargs)
File "E:\gcp-pubsub-cs\lib\site-packages\google\api_core\grpc_helpers.py", line 69, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "<string>", line 3, in raise_from
google.api_core.exceptions.ServiceUnavailable: 503 failed to connect to all addresses
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "E:\gcp-pubsub-cs\lib\site-packages\google\cloud\pubsub_v1\publisher\_batch\thread.py", line 268, in _commit
response = self._client.api.publish(
File "E:\gcp-pubsub-cs\lib\site-packages\google\pubsub_v1\services\publisher\client.py", line 613, in publish
response = rpc(request, retry=retry, timeout=timeout, metadata=metadata,)
File "E:\gcp-pubsub-cs\lib\site-packages\google\api_core\gapic_v1\method.py", line 145, in __call__
return wrapped_func(*args, **kwargs)
File "E:\gcp-pubsub-cs\lib\site-packages\google\api_core\retry.py", line 285, in retry_wrapped_func
return retry_target(
File "E:\gcp-pubsub-cs\lib\site-packages\google\api_core\retry.py", line 203, in retry_target
six.raise_from(
File "<string>", line 3, in raise_from
google.api_core.exceptions.RetryError: Deadline of 60.0s exceeded while calling functools.partial(<function _wrap_unary_errors.<locals>.error_remapped_callable at 0x000002A99F2BF0D0>, topic: "projects/stream-analytics-21/topics/twitter-stream"
messages {
data: "0"
}
messages {
data: "1"
}
messages {
data: "2"
}
messages {
data: "3"
}
messages {
data: "4"
}
messages {
data: "5"
}
messages {
data: "6"
}
messages {
data: "7"
}
messages {
data: "8"
}
messages {
data: "9"
}
, metadata=[('x-goog-request-params', 'topic=projects/stream-analytics-21/topics/twitter-stream'), ('x-goog-api-client', 'gl-python/3.9.6 grpc/1.39.0 gax/1.31.0 gccl/2.6.1')]), last exception: 503 failed to connect to all addresses
Please handle Deadline of 60.0s exceeded while calling functools.partial(<function _wrap_unary_errors.<locals>.error_remapped_callable at 0x000002A99F2BF0D0>, topic: "projects/stream-analytics-21/topics/twitter-stream"
Thank you in advance!!
Posting the answer as wiki. As per discussion of #Lauren and #Ranaveer, the issue was fixed when GOOGLE_APPLICATION_CREDENTIALS was set on the current session.

403 Request had insufficient authentication scopes. When making requests to google cloud NLP API with Application Default Credentials

I need to make requests from a python script which is hosted on an already created Google Cloud Compute engine instance, to the google cloud NLP API. So i followed the steps that were needed to accomplish this task.
I activated the NLP API for my project which also includes the VM instance
I downloaded the service key(as JSON) for the default service account that was created with the VM(Because im trying to authenticate with Application Default Credentials)
I set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the downloaded keys path
Installed google cloud client library for python
I then proceeded to check if the credentials work by running the example with the google storage API and it returned an empty list which means it worked as intended.
But i get the following error when trying to make calls to the NLP API
Traceback (most recent call last):
File "/home/user/anaconda3/lib/python3.7/site-
packages/google/api_core/grpc_helpers.py", line 57, in
error_remapped_callable
return callable_(*args, **kwargs)
File "/home/user/anaconda3/lib/python3.7/site-
packages/grpc/_channel.py", line 549, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/user/anaconda3/lib/python3.7/site-
packages/grpc/_channel.py", line 466, in _end_unary_response_blocking
raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.PERMISSION_DENIED
details = "Request had insufficient authentication scopes."
debug_error_string = "
{"created":"#1552931720.241957477","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1039,"grpc_message":"Request had insufficient authentication scopes.","grpc_status":7}">
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/user/anaconda3/lib/python3.7/site-
packages/google/cloud/language_v1/gapic/language_service_client.py",
line 230, in analyze_sentiment
request, retry=retry, timeout=timeout, metadata=metadata
File "/home/user/anaconda3/lib/python3.7/site-
packages/google/api_core/gapic_v1/method.py", line 143, in __call__
return wrapped_func(*args, **kwargs)
File "/home/user/anaconda3/lib/python3.7/site-
packages/google/api_core/retry.py", line 270, in retry_wrapped_func
on_error=on_error,
File "/home/user/anaconda3/lib/python3.7/site-
packages/google/api_core/retry.py", line 179, in retry_target
return target()
File "/home/usere/anaconda3/lib/python3.7/site-
packages/google/api_core/timeout.py", line 214, in func_with_timeout
return func(*args, **kwargs)
File "/home/user/anaconda3/lib/python3.7/site-
packages/google/api_core/grpc_helpers.py", line 59, in
error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "<string>", line 3, in raise_from
google.api_core.exceptions.PermissionDenied: 403 Request had
insufficient authentication scopes.
Any Idea why this is happening?

Python request to API keeps returning ZeroReturnError exception

Python 2.7.3
Calling an API from a Raspberry Pi 3, the API logs show it hits the correct endpoint and returns with a 200 status code, but the python code from the Pi spits out a huge error stack. I saw in some forums that the ZeroReturnError is always thrown meaning that there was nothing wrong, but that seems weird since I can't actually get the results of the response in an except block from the try.
My code is literally
import requests
response = requests.get(<URL I AM USING>, json={JSON I AM USING})
Not sure what to do.
Traceback (most recent call last):
File "music.py", line 13, in <module>
response = requests.get(url, json={'blah':{'blah':'*********'}})
File "/usr/lib/python2.7/dist-packages/requests/api.py", line 60, in get
return request('get', url, **kwargs)
File "/usr/lib/python2.7/dist-packages/requests/api.py", line 49, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 457, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 606, in send
r.content
File "/usr/lib/python2.7/dist-packages/requests/models.py", line 724, in content
self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()
File "/usr/lib/python2.7/dist-packages/requests/models.py", line 653, in generate
for chunk in self.raw.stream(chunk_size, decode_content=True):
File "/usr/lib/python2.7/dist-packages/urllib3/response.py", line 256, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "/usr/lib/python2.7/dist-packages/urllib3/response.py", line 186, in read
data = self._fp.read(amt)
File "/usr/lib/python2.7/httplib.py", line 602, in read
s = self.fp.read(amt)
File "/usr/lib/python2.7/socket.py", line 380, in read
data = self._sock.recv(left)
File "/usr/lib/python2.7/dist-packages/urllib3/contrib/pyopenssl.py", line 188, in recv
data = self.connection.recv(*args, **kwargs)
OpenSSL.SSL.ZeroReturnError
Some more searching brought me to think it was version issues.
Ran sudo pip install urllib3 --upgrade on the Raspberry Pi and it cleared it up.
I am getting a DependencyWarning about installing PySocks, but its working correctly now.

TypeError when using botocore to read from AWS SQS queue

I'm using a Tornado server with tornado-botocore to connect to Amazon SQS services.
When running stress tests we sometimes get the following exception:
Traceback (most recent call last):
File "/home/app/handlers/WebSocketsHandler.py", line 95, in listen_outgoing_queue
message = yield tornado.gen.Task(self.outgoing_queue.read)
File "/home/local/lib/python2.7/site-packages/tornado/gen.py", line 870, in run
value = future.result()
File "/home/local/lib/python2.7/site-packages/tornado/concurrent.py", line 215, in result
raise_exc_info(self._exc_info)
File "/home/local/lib/python2.7/site-packages/tornado/stack_context.py", line 314, in wrapped
ret = fn(*args, **kwargs)
File "/home/local/lib/python2.7/site-packages/tornado_botocore/base.py", line 70, in prepare_response
response_dict, operation_model.output_shape)
File "/home/local/lib/python2.7/site-packages/botocore/parsers.py", line 155, in parse
return self._do_error_parse(response, shape)
File "/home/.env/local/lib/python2.7/site-packages/botocore/parsers.py", line 314, in _do_error_parse
root = self._parse_xml_string_to_dom(xml_contents)
File "/home/local/lib/python2.7/site-packages/botocore/parsers.py", line 274, in _parse_xml_string_to_dom
parser.feed(xml_string)
TypeError: must be string or read-only buffer, not None
could it be caused by the concurrency?
has anyone encountered such behavior?
We are using tornado 4.2.1, botocore 0.65.0 and tonado-botocore 0.1.6
problem solved once i removed the #tornado.gen.engine decorator from the method