Unable to access cloudbilling.googleapis.com from GKE pod/GCE VM - google-cloud-platform

We are currently seeing an issue in terraform pipelines running in GKE pods where the datasource internally calling the cloudbilling.googleapis.com returns "Your client does not have permission to get URL".
We have added the serviceaccount to the billing account with Billing Account Viewer permission but still seeing the same behaviour(The service account has all other required permissions). A simple curl request to the cloud billing API also returns the same error and we observed this error in GKE Pod & GCP VM as well.
Previously we were running the terraform code using cloud build private worker pools and were able to access all the API's without any issues.
Request to the API without the authentication header(observed the same behaviour with auth token as well)
root#istio-ingressgateway-5944b79fdc-9fp67:/# curl https://cloudbilling.googleapis.com/v1/projects/test-projet/billingInfo
<!DOCTYPE html>
<html lang=en>
<meta charset=utf-8>
<meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
<title>Error 403 (Forbidden)!!1</title>
<style>
*{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}#media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}#media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}#media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
</style>
<a href=//www.google.com/><span id=logo aria-label=Google></span></a>
<p><b>403.</b> <ins>That’s an error.</ins>
<p>Your client does not have permission to get URL <code>/v1/projects/test-project/billingInfo</code> from this server. <ins>That’s all we know.</ins>
Calls to other API's are working as expected
root#istio-ingressgateway-5944b79fdc-9fp67:/# curl https://monitoring.googleapis.com/v3/projects/test-project/notificationChannels/6321545542211742323
{
"error": {
"code": 401,
"message": "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
"status": "UNAUTHENTICATED",
"details": [
{
"#type": "type.googleapis.com/google.rpc.ErrorInfo",
"reason": "CREDENTIALS_MISSING",
"domain": "googleapis.com",
"metadata": {
"method": "google.monitoring.v3.NotificationChannelService.GetNotificationChannel",
"service": "monitoring.googleapis.com"
}
}
]
}
}
root#istio-ingressgateway-5944b79fdc-9fp67:/#
We are using VPC SC and Google private access is enabled on the subnets. Did anyone face this issue and what is the recommended steps/config to resolve this?

We are able to resolve this issue after configuring the cloud DNS response policy for cloudbilling.googleapis.com, The previous private google access setup used the restricted.googleapis.com which is not supporting cloudbilling.googleapis.com.
We had to remove the private zone setup for the google private access and configure response policies for cloudbilling.googleapis.com and *.googleapis.com
Attached are the screenshots of the response policy configuration and the setup additional response policy rules for the API's that are not covered under SC.
Note: The terraform google_dns_response_policy_rule is currently not supporting passthrough behaviour so had to use the localdata, you can able to use passthrough behaviour for the API's once this issue is fixed https://github.com/hashicorp/terraform-provider-google/issues/11193

Related

Java - Getting "Not authorized to access the application ID" error in Google Workspace Marketplace API

I have a Google Workspace Marketplace app and want to determine which domains have installed or uninstalled it from my web app (written in Java, deployed on GAE).
I've enabled both GSuite Marketplace API and Google Workspace Marketplace SDK from the console. I'm using the same GCP project I've used to list my marketplace app while calling the relevant google service. But I keep getting the "Not authorized to access the application ID" 403 error. Here is the code:
GoogleCredential googleCredential = new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(jsonFactory)
.setServiceAccountPrivateKey(SA_PRIVATE_KEY)
.setServiceAccountId(SA_ID)
.setServiceAccountScopes(Collections.singletonList("https://www.googleapis.com/auth/appsmarketplace.license"))
.build();
GenericUrl url = new GenericUrl("https://appsmarket.googleapis.com/appsmarket/v2/licenseNotification/" + MY_APPLICATION_ID);
HttpRequestFactory httpRequestFactory = httpTransport.createRequestFactory(googleCredential);
HttpResponse httpResponse = httpRequestFactory.buildGetRequest(url).execute();
return httpResponse.parseAsString();
I tried using the default service account with no luck. The documentation doesn't explain much either. What am I missing? Can anyone point me in the right direction?
error:
{
"error": {
"code": 403,
"message": "Not authorized to access the application ID",
"errors": [
{
"message": "Not authorized to access the application ID",
"domain": "global",
"reason": "forbidden"
}
]
}
}

Code 403 : “The request is missing a valid API key.” PERMISSION_DENIED Service Account Key Google Cloud speech to text api in python"

I want to implement the Google Cloud speech to text using a service account. What i have try is i have set the environment variable to that json and send the post request to this url 'https://speech.googleapis.com/v1/speech:longrunningrecognize'.
Code:
req = requests.post(url, data={
"audio":{
"content":enc
},
"config":{
"audioChannelCount":2,
"enableSeparateRecognitionPerChannel":True,
"enableWordTimeOffsets":True,
"diarizationConfig":{
"enableSpeakerDiarization": True,
"minSpeakerCount": 1,
"maxSpeakerCount": 2
},
}})
Error:
403
{
"error": {
"code": 403,
"message": "The request is missing a valid API key.",
"status": "PERMISSION_DENIED"
}
}
The error message indicates that you are not authenticating correctly. The way to do this is to pass an authentication token as a Bearer Token header in your request.
The following documentation explains how to generate the required credentials and pass them with the request, this provides an overview of service accounts Service accounts overview
Creating a service account instructions Creating service accounts
Once you have created the service account you generate the credentials which are stored in json format, these are then passed as a Bearer Token

Google Cloud Run Authentication Service-to-Service

I have two services (APIs) deployed on GCP Cloud Run. Call them service-one.myDomain.com and service-two.myDomain.com. I would like service-one to be authenticated in calling service-two independently of what any user is doing.
I've read and implemented the instructions from GCP Cloud Run docs on Authenticating service-to-service (https://cloud.google.com/run/docs/authenticating/service-to-service) but service-one.myDomain.com is unsuccessful in calling service-two.myDomain.com receiving a 401:Unauthorized response.
Any thoughts on how to get service-one to successfully call service-two?
Here's my setup:
IAM and service accounts:
On google IAM, I created two service accounts and granted them both the "Cloud Run Invoker" (roles/run.invoker) role:
service-one#myproject.iam.gserviceaccount.com
service-two#myproject.iam.gserviceaccount.com
Inside Cloud Run I changed the service account from the "Default compute service account" to the service accounts I created. I assigned service-one#myproject.iam.gserviceaccount.com for service-one.myDomain.com and service-two#myproject.iam.gserviceaccount.com for service-two.myDomain.com
OIDC Auth token:
In service-one.myDomain.com I make a call to the metadata server to get a token (jwt) from the following url:
http://metadata/computeMetadata/v1/instance/service-accounts/default/identity?audience=https://service-two.myDomain.com with a request header set as {'Metadata-Flavor': 'Google'} The request is successful and the token I receive is decoded to have the following payload:
{
"alg": "RS256",
"kid": "9cef5340642b157fa8a4f0d874fe7543872d82db",
"typ": "JWT"
}
{
"aud": "https://service-two.mydomain.com",
"azp": "100959068407876085761",
"email": "service-one#myproject.iam.gserviceaccount.com",
"email_verified": true,
"exp": 1572806540,
"iat": 1572802940,
"iss": "https://accounts.google.com",
"sub": "100953168404568085761"
}
Http request:
Using the token I make a request from service-one.myDomain.com to an http endpoint on service-two.myDomain.com. I set the request header with {'Authorization': 'Bearer {token}'} ({token} is value of token).
Http Response:
The response is a 401 Unauthorized and my logs show the response headers to include:
{'WWW-Authenticate': 'Bearer error="invalid_token" error_description="The access token could not be verified"'}
With a content of:
"
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>401 Unauthorized</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Unauthorized</h1>
<h2>Your client does not have permission to the requested URL <code>/health</code>.</h2>
<h2></h2>
</body></html>
"
I'm stumped.... any ideas on what I'm missing to get service-one to authenticate to service-two?
The answer was to use the gcp cloud run generated Url as the audience in the OIDC token request. And relatedly the "aud" field in the jwt.
My discovery was that Service-to-Service authentication in cloud run does not support custom domains (myDomain.com). I was using my custom domain.
(I feel like a bonehead) thanks #guillaumeblaquiere

401 unauthorized error when opening the URL of my Google Cloud Run service

I built a sample container(sample of Go, https://cloud.google.com/run/docs/quickstarts/build-and-deploy) and deployed to cloud run (I unchecked "allow unauthenticated invocations").
However, when I open the endpoint URL of my service, I get a 401 page,
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>401 Unauthorized</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Unauthorized</h1>
<h2>Your client does not have permission to the requested URL <code>/</code>.</h2>
<h2></h2>
</body></html>
According to the official document (https://cloud.google.com/run/docs/securing/authenticating#developers), I used this command.
curl -H "Authorization: Bearer $(gcloud config config-helper --format 'value(credential.id_token)')" [SERVICE_URL]
My user account has "roles/run.invoker"
When I checked "allow unauthenticated invocations", I got an expected result.
Is there anything to open the endpoint?
Please make sure gcloud version is at least 243.0.0 by "gcloud --version"
Please use "gcloud components update" to update if gcloud is too old
[Updated] Using gcloud in Cloud Shell to call authentication-required Cloud Run service is also working now.
Thanks!
I managed to connect from a cloud function, finally, after 15-minute wrestling with the same 401 error when invoking a cloud run service that does not allow unauthenticated invocations:
import google.auth
auth_req = google.auth.transport.requests.Request()
id_token = google.oauth2.id_token.fetch_id_token(auth_req, URL)
headers = {'Authorization': 'Bearer ' + id_token,
"Content-Type": "application/json; charset=utf-8"}
response = requests.post(URL, json=request_json, headers=headers)

Delete DynamoDB entries with AWS API

I am learning AWS API Gateway to manage the api in the cloud and not through express.js. I created the method DELETE under a resource with the following Integration Request.
Integration Request
Integration Type: AWS Service
AWS Region: us-east-1
AWS Service: DynamoDB
HTTP method: DELETE
Action: deleteItem
Execution role: XXXXXX
Content Handling: Passthrough
the execution role is working because i can read/write to DynamoDB. a Body Mapping Template that has a Content-Type: application/json as follows:
{
"TableName": "comments",
"Key": {
"commentId": {
"S": "$context.requestId"
},
"pageId": {
"S": "$input.path('$.pageId')"
},
"userName": {
"S": "$input.path('$.userName')"
}
}
}
when i try to run the test with a Request Body as follows:
{
"pageId": "breaking-news-story-01-18-2016",
"userName": "cacevedo"
}
I get the following information:
Request: /comments
Status: 200
Latency: 80 ms
Response Body
Response Body:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html
xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head>
<title>Page Not Found</title> </head> <body>Page Not Found</body>
</html>
Can someone help me out with this, because i am not able to delete the item. yet it seems that the integration request is working. Am i missing something or doing something wrong?
Just in case anyone runs into the same problem. the HTTP verb needs to be a post and not delete and also is case sensitive so instead of using deleteItem. use DeleteItem