I'm trying to build small function (which later deployed to cloud function) to restore BAK file to cloud sql. This function will be trigerred by cron.
I'm kind of lost when reading the docs about authorize this API: https://cloud.google.com/sql/docs/sqlserver/import-export/importing#importing_data_from_a_bak_file_in
Already create service account which include this role: Cloud SQL Admin, Storage Admin, Storage Object Admin, Storage Object Viewer and choose that Service Account from dropdown when creating Cloud Function but not work.
Also tried generating API keys after reading this: https://cloud.google.com/sql/docs/sqlserver/admin-api/how-tos/authorizing
So my POST url became this:
https://www.googleapis.com/sql/v1beta4/projects/project-id/instances/instance-id/import?key=generatedAPIKey
but still got an error:
"error": {
"code": 401,
"message": "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
"errors": [
{
"message": "Login Required.",
"domain": "global",
"reason": "required",
"location": "Authorization",
"locationType": "header"
}
],
"status": "UNAUTHENTICATED"
}
}
Do I need to use Oauth 2 for this? This is my code in Cloud Function:
import http.client
import mimetypes
def restore_bak(request):
conn = http.client.HTTPSConnection("www.googleapis.com")
payload = "{\r\n \"importContext\":\r\n {\r\n \"fileType\": \"BAK\",\r\n \"uri\": \"gs://{bucket_name}/{backup_name}.bak\",\r\n \"database\": \"{database_name}\"\r\n }\r\n}\r\n"
headers = {
'Content-Type': 'application/json'
}
conn.request("POST", "/sql/v1beta4/projects/{project_id}/instances/{instance_name}/import", payload, headers)
res = conn.getresponse()
data = res.read()
print(data.decode("utf-8"))
return(data.decode("utf-8"))
This looks like python, so I would recommend using the Discovery Client Library for Python. This library provides a convient wrapper around the SQL Admin API:
# Construct the service object for the interacting with the Cloud SQL Admin API.
service = discovery.build('sqladmin', 'v1beta4', http=http)
req = service.instances().list(project="PROJECT_ID")
resp = req.execute()
print json.dumps(resp, indent=2)
By default, this library uses the "Application Default Credentials (ADC)" strategy to obtain your credentials from the environment.
You can also manually authenticate your requests (for example, if you want to use asyncio) by creating an oauth2 token and setting it as a header in your request. The easiest way to do this is to use the google-auth package to get the ADC and set it as a header:
import google.auth
import google.auth.transport.requests
credentials, project_id = google.auth.default()
credentials.refresh(google.auth.transport.requests.Request())
headers = {
"Authorization": "Bearer {}".format(credentials.token),
"Content-Type": "application/json"
}
Related
I am trying to refresh a PBI Data Flow using an ADF web activity by authenticating using the data factory's Managed Identity.
Here is my input to the activity:
{
"url": "https://api.powerbi.com/v1.0/myorg/groups/1dec5b21-ba60-409b-80cb-de61272ee504/dataflows/0e256da2-8823-498c-b779-3e7a7568137f/refreshes",
"connectVia": {
"referenceName": "My-AzureVM-IR",
"type": "IntegrationRuntimeReference"
},
"method": "POST",
"headers": {
"Content-Type": "application/json",
"User-Agent": "AzureDataFactoryV2",
"Host": "api.powerbi.com",
"Accept": "*/*",
"Connection": "keep-alive"
},
"body": "{\"notifyOption\":\"MailOnFailure\"}",
"disableCertValidation": true,
"authentication": {
"type": "MSI",
"resource": "https://analysis.windows.net/powerbi/api"
}
}
It generates the following error when doing a debug run:
Failure type: User configuration issue
Details: {"error":{"code":"InvalidRequest","message":"Unexpected dataflow error: "}}
I have tried this exact URL in Postman using Bearer Token Authentication and it works. Our AAD Admin group said they added our ADF's Managed Identity to the permission list for the PBI API, so I am not sure what is going on here.
Just an FYI, I was able to get the ADF Managed Identity working with data flow refreshes using the HTTP request in my original post.
The key was after having the Tenant Admins add the Managed Identity to a security group with API access, I then also had to add the Managed Identity to the PBI Workspace access list as a Member.
Then my API call worked from ADF using the MSI. No Bearer login token needed.
I am trying to fetch the data studio assets to manage the permissions based on that data through APIs.
I am using oAuth2 access token generated using the service account as I want to automate this process and not rely on user consent for authorization every time. oAuth2 authorization using service account
Steps I have followed:
Created service account in Google cloud and enabled Google Workspace Domain-wide Delegation
Delegating domain-wide authority to the service account through Google Workspace account
For the following scopes:
https://www.googleapis.com/auth/datastudio
https://www.googleapis.com/auth/userinfo.email
https://www.googleapis.com/auth/userinfo.profile
openid
Created and signed JWT
Used JWT token to get the oAuth2 access token to make Datastudio API calls.
Using the following snippet to generate the signed JWT.
import jwt
import time
import json
import requests
iat = int(time.time())
exp = iat + 3600
claim_set = {"iss": "datastudio-manager#data-project.iam.gserviceaccount.com",
"scope": "https://www.googleapis.com/auth/datastudio https://www.googleapis.com/auth/datastudio.readonly https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/userinfo.profile openid",
"aud": "https://oauth2.googleapis.com/token", "exp": exp, "iat": iat}
encoded = jwt.encode(claim_set, private_key, algorithm="RS256")
response = requests.post("https://oauth2.googleapis.com/token", params={
"grant_type": "urn:ietf:params:oauth:grant-type:jwt-bearer",
"assertion": encoded
})
print(response.json()["access_token"])
Using token generated from above step to make API call.
curl -H "Authorization: Bearer <access_token>" https://datastudio.googleapis.com/v1/assets:search?assetTypes=report
Response of the above request is {} with status 200 and when I am trying to view permissions for a particular asset it is giving me the following response.
API endpoint: https://datastudio.googleapis.com/v1/assets/<asset_id>/permissions
{
"error": {
"code": 403,
"message": "The caller does not have permission",
"status": "PERMISSION_DENIED"
}
}
Is authentication using access token generated through the above oAuth2 method supported for Data studio APIs?
Any leads are much appreciated, thanks in advance!
I have a Google Workspace Marketplace app and want to determine which domains have installed or uninstalled it from my web app (written in Java, deployed on GAE).
I've enabled both GSuite Marketplace API and Google Workspace Marketplace SDK from the console. I'm using the same GCP project I've used to list my marketplace app while calling the relevant google service. But I keep getting the "Not authorized to access the application ID" 403 error. Here is the code:
GoogleCredential googleCredential = new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(jsonFactory)
.setServiceAccountPrivateKey(SA_PRIVATE_KEY)
.setServiceAccountId(SA_ID)
.setServiceAccountScopes(Collections.singletonList("https://www.googleapis.com/auth/appsmarketplace.license"))
.build();
GenericUrl url = new GenericUrl("https://appsmarket.googleapis.com/appsmarket/v2/licenseNotification/" + MY_APPLICATION_ID);
HttpRequestFactory httpRequestFactory = httpTransport.createRequestFactory(googleCredential);
HttpResponse httpResponse = httpRequestFactory.buildGetRequest(url).execute();
return httpResponse.parseAsString();
I tried using the default service account with no luck. The documentation doesn't explain much either. What am I missing? Can anyone point me in the right direction?
error:
{
"error": {
"code": 403,
"message": "Not authorized to access the application ID",
"errors": [
{
"message": "Not authorized to access the application ID",
"domain": "global",
"reason": "forbidden"
}
]
}
}
I want to implement the Google Cloud speech to text using a service account. What i have try is i have set the environment variable to that json and send the post request to this url 'https://speech.googleapis.com/v1/speech:longrunningrecognize'.
Code:
req = requests.post(url, data={
"audio":{
"content":enc
},
"config":{
"audioChannelCount":2,
"enableSeparateRecognitionPerChannel":True,
"enableWordTimeOffsets":True,
"diarizationConfig":{
"enableSpeakerDiarization": True,
"minSpeakerCount": 1,
"maxSpeakerCount": 2
},
}})
Error:
403
{
"error": {
"code": 403,
"message": "The request is missing a valid API key.",
"status": "PERMISSION_DENIED"
}
}
The error message indicates that you are not authenticating correctly. The way to do this is to pass an authentication token as a Bearer Token header in your request.
The following documentation explains how to generate the required credentials and pass them with the request, this provides an overview of service accounts Service accounts overview
Creating a service account instructions Creating service accounts
Once you have created the service account you generate the credentials which are stored in json format, these are then passed as a Bearer Token
As per the google documentation we ended up creating a service account (since there’s no user intervention and it’s middleware invoking the google api) and associating with the google admin api; We’re able to get a token back which we’re sending to the google api but here’s the response we’re seeing:
GET https://www.googleapis.com/admin/directory/v1/customer/my_customer /devices/chromeos?access_token=y000000000898999(mock token digits)
Response: {
"error": {
"errors": [
{
"domain": "global",
"reason": "invalid",
"message": "Invalid Input: Domain ClientUserInfo is not present"
}
],
"code": 400,
"message": "Invalid Input: Domain ClientUserInfo is not present"
}
}
Any ideas? My thought is that it’s to do with the my_customer as the id.
Using my_customer as the id should work but in this case, since the service account is not related to a domain, you are getting the error.
Admin SDK should be access by the admin of the domain. So in this case if you are using a service account, you should impersonate the domain admin account in order to make a valid call on his behalf.
You can also try this method from the documentation, accessing it as the admin of the domain.