I am trying to fetch the data studio assets to manage the permissions based on that data through APIs.
I am using oAuth2 access token generated using the service account as I want to automate this process and not rely on user consent for authorization every time. oAuth2 authorization using service account
Steps I have followed:
Created service account in Google cloud and enabled Google Workspace Domain-wide Delegation
Delegating domain-wide authority to the service account through Google Workspace account
For the following scopes:
https://www.googleapis.com/auth/datastudio
https://www.googleapis.com/auth/userinfo.email
https://www.googleapis.com/auth/userinfo.profile
openid
Created and signed JWT
Used JWT token to get the oAuth2 access token to make Datastudio API calls.
Using the following snippet to generate the signed JWT.
import jwt
import time
import json
import requests
iat = int(time.time())
exp = iat + 3600
claim_set = {"iss": "datastudio-manager#data-project.iam.gserviceaccount.com",
"scope": "https://www.googleapis.com/auth/datastudio https://www.googleapis.com/auth/datastudio.readonly https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/userinfo.profile openid",
"aud": "https://oauth2.googleapis.com/token", "exp": exp, "iat": iat}
encoded = jwt.encode(claim_set, private_key, algorithm="RS256")
response = requests.post("https://oauth2.googleapis.com/token", params={
"grant_type": "urn:ietf:params:oauth:grant-type:jwt-bearer",
"assertion": encoded
})
print(response.json()["access_token"])
Using token generated from above step to make API call.
curl -H "Authorization: Bearer <access_token>" https://datastudio.googleapis.com/v1/assets:search?assetTypes=report
Response of the above request is {} with status 200 and when I am trying to view permissions for a particular asset it is giving me the following response.
API endpoint: https://datastudio.googleapis.com/v1/assets/<asset_id>/permissions
{
"error": {
"code": 403,
"message": "The caller does not have permission",
"status": "PERMISSION_DENIED"
}
}
Is authentication using access token generated through the above oAuth2 method supported for Data studio APIs?
Any leads are much appreciated, thanks in advance!
Related
I have a wso2 apim 3.2 setup up with wso2km 5.10. I have configured the Apim to pass Enduser attributes to the backend but cannot get the role claim returned. apim and the km manager are on separate machines. I seem to get just the standard claims returned. I have enable the required sections of the deployment.toml and I'm not seeing what I have wrong any help would be appreciated.
[apim.jwt]<br/>
enable = true<br/>
claim_dialect = "http://wso2.org/claims"<br/>
claims_extractor_impl = "org.wso2.carbon.apimgt.impl.token.ExtendedDefaultClaimsRetriever"
here it what is returned.
{<br/>
"http://wso2.org/claims/apiname": "xxxxxxxx",<br/>
"http://wso2.org/claims/applicationtier": "Unlimited",<br/>
"http://wso2.org/claims/version": "1.0.0",<br/>
"http://wso2.org/claims/keytype": "PRODUCTION",<br/>
"iss": "wso2.org/products/am",<br/>
"http://wso2.org/claims/applicationname": "xxxxxx",<br/>
"http://wso2.org/claims/enduser": "xxxxxx",<br/>
"http://wso2.org/claims/enduserTenantId": "-1234",<br/>
"http://wso2.org/claims/applicationUUId": "348d1ff9-06f5-4f3f-aa94-83f32f4a1f2a",<br/>
"http://wso2.org/claims/subscriber": "xxxxxxx",<br/>
"azp": "NjYtixQB4VbFLeunCrj1U1ZYcfga",<br/>
"http://wso2.org/claims/tier": "Unlimited",<br/>
"scope": "openid",<br/>
"exp": 1601500346,<br/>
"http://wso2.org/claims/applicationid": "8",<br/>
"http://wso2.org/claims/usertype": "Application_User",<br/>
"http://wso2.org/claims/apicontext": "/xxxxxxxxxxx"<br/>
}
{
"sub": "admin#carbon.super",
"aud": "eZi3HFaydfnHtlZRZDpzuz6N5pMa",
"nbf": 1602022037,
"azp": "eZi3HFaydfnHtlZRZDpzuz6N5pMa",
"scope": "am_application_scope default",
"iss": "https://xxxxxxxxxxxxxxxx",
"exp": 1602025637,
"iat": 1602022037,
"jti": "7845227d-6800-4ff2-9982-3d338e45abb6"
}
There are two ways to include user claims to the backend JWT
Implement custom token generator
Adding required claims to the JWT access token
Adding required claims to the JWT access token
With APIM 3.2.0 it supports only JWT access token for the new application it registers. To include any user claims to backend JWT, the required claims should be in the JWT access token since GW is responsible to generate backend JWT.
To include user claims to the JWT access token follow the below steps.
Identify the service provider for the application from the management console
Edit the service provide and configure requested claims under the Claim Configuration menu
Generate an access token with openid scope
curl -k -X POST https://localhost:8243/token -d
"grant_type=client_credentials&scope=openid" -H"Authorization: Basic
VEJEMXJZazZpSWVlaTlnVzRNTENBYXNEQW9JYTpkRnJ0bVJjaklqUUtkSVVYeVY4aWxlZjBQNWdh"
An access token will be issued with the requested claims
{
"sub": "admin#carbon.super",
"aud": "TBD1rYk6iIeei9gW4MLCAasDAoIa",
"nbf": 1602047260,
"azp": "TBD1rYk6iIeei9gW4MLCAasDAoIa",
"scope": "am_application_scope openid",
"iss": "https://localhost:9443/oauth2/token",
"groups": [
"Internal/subscriber",
"Internal/creator",
"Application/apim_devportal",
"Application/admin_sample_PRODUCTION",
"Internal/publisher",
"Internal/everyone",
"Internal/devops",
"Application/apim_admin_portal",
"Application/admin_key1_PRODUCTION",
"admin",
"Internal/analytics",
"Application/apim_publisher"
],
"exp": 1602050860,
"iat": 1602047260,
"jti": "d74a617e-e976-42f4-8323-c1c2271d046e"
}
Access an API with the above access token and backend JWT contains the required claims.
how do I add user roles to JWT generated through OAuth2 Password Grant as described here:
I tried this approach but it adds custom claims only to JWT passed to backend but there is nothing in JWT used to authenticate clients.
What I'm trying to do is to add a login page to Angular application and call https://[APIM]/token to get token when successful authentication occurs. Roles are important to render correct menus based on user roles.
Thanks in advance,
You need to request the token with openid scope to retrieve the additional user information as claims of the JWT token. You can refer https://apim.docs.wso2.com/en/latest/learn/api-security/openid-connect/obtaining-user-profile-information-with-openid-connect/ for more details.
For instance, if you want to get the user roles in the generated JWT, you can add the http://wso2.org/claims/role claim as a requested claim under Claim Configuration to the service provider you are using from the carbon console. Refer https://is.docs.wso2.com/en/5.10.0/learn/configuring-claims-for-a-service-provider/#claim-mapping for more details.
Then when you are invoking the token endpoint, you need to add the openid scope.
curl -k -d "grant_type=password&username=<USERNAME>&password=<PASSWORD>&scope=openid" -H "Authorization: Basic <BASE64 ENCODED CONSUMER_KEY:CONSUMER_SECRET>, Content-Type: application/x-www-form-urlencoded" https://<GATEWAY_HOSTNAME>:<PORT>/token
The generated JWT token payload will be something like this,
{
"sub": "admin",
"aut": "APPLICATION_USER",
"aud": "5af6EfSzqxS_dfmUnQ28sHdpZzYa",
"nbf": 1610395871,
"azp": "5af6EfSzqxS_dfmUnQ28sHdpZzYa",
"scope": "openid",
"iss": "https://localhost:9443/oauth2/token",
"groups": [
"Internal/subscriber",
"Internal/creator",
"Application/admin_DefaultApplication_PRODUCTION",
"Application/apim_devportal",
"Internal/publisher",
"Internal/everyone",
"Internal/devops",
"Application/apim_admin_portal",
"admin",
"Internal/analytics",
"Application/apim_publisher"
],
"exp": 1610399471,
"iat": 1610395871,
"jti": "75ddfca2-5088-435d-825a-3320efc10036"
}
Hope this helped!
I'm trying to build small function (which later deployed to cloud function) to restore BAK file to cloud sql. This function will be trigerred by cron.
I'm kind of lost when reading the docs about authorize this API: https://cloud.google.com/sql/docs/sqlserver/import-export/importing#importing_data_from_a_bak_file_in
Already create service account which include this role: Cloud SQL Admin, Storage Admin, Storage Object Admin, Storage Object Viewer and choose that Service Account from dropdown when creating Cloud Function but not work.
Also tried generating API keys after reading this: https://cloud.google.com/sql/docs/sqlserver/admin-api/how-tos/authorizing
So my POST url became this:
https://www.googleapis.com/sql/v1beta4/projects/project-id/instances/instance-id/import?key=generatedAPIKey
but still got an error:
"error": {
"code": 401,
"message": "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
"errors": [
{
"message": "Login Required.",
"domain": "global",
"reason": "required",
"location": "Authorization",
"locationType": "header"
}
],
"status": "UNAUTHENTICATED"
}
}
Do I need to use Oauth 2 for this? This is my code in Cloud Function:
import http.client
import mimetypes
def restore_bak(request):
conn = http.client.HTTPSConnection("www.googleapis.com")
payload = "{\r\n \"importContext\":\r\n {\r\n \"fileType\": \"BAK\",\r\n \"uri\": \"gs://{bucket_name}/{backup_name}.bak\",\r\n \"database\": \"{database_name}\"\r\n }\r\n}\r\n"
headers = {
'Content-Type': 'application/json'
}
conn.request("POST", "/sql/v1beta4/projects/{project_id}/instances/{instance_name}/import", payload, headers)
res = conn.getresponse()
data = res.read()
print(data.decode("utf-8"))
return(data.decode("utf-8"))
This looks like python, so I would recommend using the Discovery Client Library for Python. This library provides a convient wrapper around the SQL Admin API:
# Construct the service object for the interacting with the Cloud SQL Admin API.
service = discovery.build('sqladmin', 'v1beta4', http=http)
req = service.instances().list(project="PROJECT_ID")
resp = req.execute()
print json.dumps(resp, indent=2)
By default, this library uses the "Application Default Credentials (ADC)" strategy to obtain your credentials from the environment.
You can also manually authenticate your requests (for example, if you want to use asyncio) by creating an oauth2 token and setting it as a header in your request. The easiest way to do this is to use the google-auth package to get the ADC and set it as a header:
import google.auth
import google.auth.transport.requests
credentials, project_id = google.auth.default()
credentials.refresh(google.auth.transport.requests.Request())
headers = {
"Authorization": "Bearer {}".format(credentials.token),
"Content-Type": "application/json"
}
I want to implement the Google Cloud speech to text using a service account. What i have try is i have set the environment variable to that json and send the post request to this url 'https://speech.googleapis.com/v1/speech:longrunningrecognize'.
Code:
req = requests.post(url, data={
"audio":{
"content":enc
},
"config":{
"audioChannelCount":2,
"enableSeparateRecognitionPerChannel":True,
"enableWordTimeOffsets":True,
"diarizationConfig":{
"enableSpeakerDiarization": True,
"minSpeakerCount": 1,
"maxSpeakerCount": 2
},
}})
Error:
403
{
"error": {
"code": 403,
"message": "The request is missing a valid API key.",
"status": "PERMISSION_DENIED"
}
}
The error message indicates that you are not authenticating correctly. The way to do this is to pass an authentication token as a Bearer Token header in your request.
The following documentation explains how to generate the required credentials and pass them with the request, this provides an overview of service accounts Service accounts overview
Creating a service account instructions Creating service accounts
Once you have created the service account you generate the credentials which are stored in json format, these are then passed as a Bearer Token
I have two services (APIs) deployed on GCP Cloud Run. Call them service-one.myDomain.com and service-two.myDomain.com. I would like service-one to be authenticated in calling service-two independently of what any user is doing.
I've read and implemented the instructions from GCP Cloud Run docs on Authenticating service-to-service (https://cloud.google.com/run/docs/authenticating/service-to-service) but service-one.myDomain.com is unsuccessful in calling service-two.myDomain.com receiving a 401:Unauthorized response.
Any thoughts on how to get service-one to successfully call service-two?
Here's my setup:
IAM and service accounts:
On google IAM, I created two service accounts and granted them both the "Cloud Run Invoker" (roles/run.invoker) role:
service-one#myproject.iam.gserviceaccount.com
service-two#myproject.iam.gserviceaccount.com
Inside Cloud Run I changed the service account from the "Default compute service account" to the service accounts I created. I assigned service-one#myproject.iam.gserviceaccount.com for service-one.myDomain.com and service-two#myproject.iam.gserviceaccount.com for service-two.myDomain.com
OIDC Auth token:
In service-one.myDomain.com I make a call to the metadata server to get a token (jwt) from the following url:
http://metadata/computeMetadata/v1/instance/service-accounts/default/identity?audience=https://service-two.myDomain.com with a request header set as {'Metadata-Flavor': 'Google'} The request is successful and the token I receive is decoded to have the following payload:
{
"alg": "RS256",
"kid": "9cef5340642b157fa8a4f0d874fe7543872d82db",
"typ": "JWT"
}
{
"aud": "https://service-two.mydomain.com",
"azp": "100959068407876085761",
"email": "service-one#myproject.iam.gserviceaccount.com",
"email_verified": true,
"exp": 1572806540,
"iat": 1572802940,
"iss": "https://accounts.google.com",
"sub": "100953168404568085761"
}
Http request:
Using the token I make a request from service-one.myDomain.com to an http endpoint on service-two.myDomain.com. I set the request header with {'Authorization': 'Bearer {token}'} ({token} is value of token).
Http Response:
The response is a 401 Unauthorized and my logs show the response headers to include:
{'WWW-Authenticate': 'Bearer error="invalid_token" error_description="The access token could not be verified"'}
With a content of:
"
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>401 Unauthorized</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Unauthorized</h1>
<h2>Your client does not have permission to the requested URL <code>/health</code>.</h2>
<h2></h2>
</body></html>
"
I'm stumped.... any ideas on what I'm missing to get service-one to authenticate to service-two?
The answer was to use the gcp cloud run generated Url as the audience in the OIDC token request. And relatedly the "aud" field in the jwt.
My discovery was that Service-to-Service authentication in cloud run does not support custom domains (myDomain.com). I was using my custom domain.
(I feel like a bonehead) thanks #guillaumeblaquiere