I have a C# App that uses a Active Directory User to perform the following operations, I am trying to convert it to use a Active Directory Service Principal. I am using Power Embedded Workspace with Premium Capacity in Azure
Create DataSet
Create Report
Refresh DataSet
However I am running into issues when trying to get the same actions performed using a Service Principal.
What have I tried
I have the following things setup
01. Tenant Settings -> Developer Options :
created app registration in Azure Active Directory
created security group in Azure Active Directory.
added Service Principal (created in step 1) to the security group (created in step 2)
As app.powerbi.com administrator enabled the developer settings -> access to admin api
restricted the access to a specific group and made sure that the Service Principal I am using is added to this security group
02. Workspace Settings : powerbi workspace -> access control
added Service Principal as "Admin" to the workspace
03. App Owns Data workflow
used the suggested github repository App Owns Data Solution
plugged in the correct values in appsettings.json
ClientID,
SecretKey
TenantID
WorkspaceId
ReportId
I am getting the error
Operation returned an invalid status code 'Unauthorized'
at Microsoft.PowerBI.Api.ReportsOperations.GetReportInGroupWithHttpMessagesAsync(Guid groupId, Guid reportId, Dictionary`2 customHeaders, CancellationToken cancellationToken)
at Microsoft.PowerBI.Api.ReportsOperationsExtensions.GetReportInGroupAsync(IReportsOperations operations, Guid groupId, Guid reportId, CancellationToken cancellationToken)
at Microsoft.PowerBI.Api.ReportsOperationsExtensions.GetReportInGroup(IReportsOperations operations, Guid groupId, Guid reportId)
at AppOwnsData.Services.PbiEmbedService.GetEmbedParams(Guid workspaceId, Guid reportId, Guid additionalDatasetId) in C:\Campuslabs\TeamFoundation\PowerBI-Developer-Samples\.NET Core\Embed for your customers\AppOwnsData\Services\PbiEmbedService.cs:line 46
at AppOwnsData.Controllers.EmbedInfoController.GetEmbedInfo() in C:\Campuslabs\TeamFoundation\PowerBI-Developer-Samples\.NET Core\Embed for your customers\AppOwnsData\Controllers\EmbedInfoController.cs:line 45
My question is
How do I troubleshoot this issue
Are Service Principal Credentials allowed to refresh / create / data sets on Power Embedded Workspace with Premium Capacity
Regarding trouble shooting, it seems from the error stack you sent that the issue lies with permissions on the workspace, double check that the Service Principal you added as an admin on the workspace is the same one you are using in your app.
Regarding the second question, the answer to that is yes. A Service Principal can refresh / create / data sets on Power Embedded Workspace with Premium Capacity.
Related
I've generated a server key in the API Manager and attempted to execute the following on my Mac:
curl 'https://sheets.googleapis.com/v4/spreadsheets/MySheetID?ranges=A1:B5&key=TheServerKeyIGeneratedInAPIManager'
But this is what it returns:
{
"error": {
"code": 403,
"message": "The caller does not have permission",
"status": "PERMISSION_DENIED"
}
}
What am I doing wrong here?
To solve this issue, try to:
Create a service account: https://console.developers.google.com/iam-admin/serviceaccounts/
In options, create a key: this key is your usual client_secret.json - use it the same way
Make the role owner for the service account (Member name = service account ID = service account email ex: thomasapp#appname-201813.iam.gserviceaccount.com
Copy the email address of your service account = service account ID
Simply go in your browser to the Google sheet you want to interact with
Go to SHARE on the top right of your screen
Go to advanced settings and share it with an email address of your service account ex: thomasapp#appname-201813.iam.gserviceaccount.com
I know it is a little late to answer but for other people struggling with the same issue.
Just change the permission of the sheet to public on your drive so it can be accessed without authentication via API calls.
To change access:
Open sheet in google drive
On top right corner, click share
On bottom of prompt window, click advanced
Change permission to public or people with link (no signin required)
Send API request to fetch data from sheets without authentication.
Note: if the sheet contains sensitive data then it is not safe to make it public and rather do it with Authenticated access.
Make sure to pay attention to #KishanPatel's comment:
Also, you can share this sheet with specific email Ex. your service
account (project) email. "client_email":
"XXXXX#northern-gasket-XXXX.iam.gserviceaccount.com", This will allow
to access sheet by your script.
Visual Simplification of the Answers:
Option 1 - Turn the file into public (if sheets the sheet contains sensitive data)
Option 2 - Share file with Service Account Email (IAM & Admin -> Service Accounts -> Details -> Email)
The easiest way is to fix using gcloud cli. More docs here https://cloud.google.com/pubsub/docs/quickstart-cli#before-you-begin
install gcloud
sudo apt-get install google-cloud-sdk
then call
gcloud init
then check your active project and credentials
gcloud config configurations list
If it is not ok, make sure you are authenticated with the correct account:
gcloud auth list
* account 1
account 2
Change to the project's account if not:
gcloud config set account `ACCOUNT`
Depending on the account, the project list will be different:
gcloud projects list
- project 1
- project 2...
Switch to intended project:
gcloud config set project `PROJECT NAME`
Then Create Application Default Credentials with gcloud auth application-default login, and then google-cloud will automatically detect such credentials.
My 10 cents... A simple example to read the sheet using Java.
private Credential getCredentials() throws IOException {
final InputStream accessKey = new ByteArrayInputStream("<credential json>");
final GoogleCredential credential = GoogleCredential.fromStream(accessKey)
.createScoped(Collections.singleton(SheetsScopes.SPREADSHEETS_READONLY));
return credential;
}
private HttpTransport httpTransport() {
try {
return GoogleNetHttpTransport.newTrustedTransport();
} catch (GeneralSecurityException | IOException e) {
throw new SpreadSheetServiceException(e);
}
}
Sheets service = new Sheets.Builder(httpTransport(), JSON_FACTORY, getCredentials())
.setApplicationName("app-name")
.build();
ValueRange response = service.spreadsheets().values()
.get("<spread_sheet_id>", "A1:A")
.execute();
In my case, solving this problem turned out to be trivial. You just have to:
Enter the google sheet that we want to remotely edit.
In the upper right corner, set - anyone who has the link can enter
Most importantly - on the right side you need to set permissions for people who have the link as 'editor'
if you still do not have permission, it means that you have to go to the website:
https://console.developers.google.com/iam-admin/iam/ then select your project, then select "Service accounts" and create a new one as role "owner" or" editor" for the project for example (or use one that already exists and click "create new key")
The "key" is a json file that will be downloaded when you create the account (or use "create new key" there).
We have a requirement to fetch data from a rest api into powerbi and schedule a refresh every night. The rest api support jwt authentication so it needs header with xapikey and access token.
I have managed to write a function in power query to get access token from our auth endpoint and able to inject access token for the rest api call and it works fine with powerbi-desktop. I have published the report to powerbi cloud.
The auth endpoint require username and password, we would not like to store this details in .pbix file and publish to cloud but instead use azure key vault and powerbi to fetch details at runtime.
Please advise ?
Power Automate has a great Azure Vault connector.
You could make a simple 3-action flow:
A post to that URL will json back the secret/credentials.
Now, here is the goofy part - hide that URL in a permissioned location (Onedrive, Sharepoint, etc). Have your pbi pickup from that location, using privileged credentials. Now the URL and the credentials get picked up at runtime, and neither is persisted in PBIX.
I am assuming that there is an available premium PAutomate env in which to spin up that flow, of course. But, given that you already have an azure vault, that seems like a standard PBI+ toolkit to have at that point.
I'm trying to get my Flask App Engine project to read data stored in Cloud SQL MySQL 5.7 database. Something has gone wrong as all I've gotten are pymysql.err.OperationalError. I've been following the instructions here: https://cloud.google.com/sql/docs/mysql/connect-app-engine.
The Cloud SQL Admin API is enabled.
According to the linked document:
App Engine uses a service account to authorize your connections to Cloud SQL. This service account must have the correct IAM permissions to successfully connect. Unless otherwise configured, the default service account is in the format service-PROJECT_NUMBER#gae-api-prod.google.com.iam.gserviceaccount.com.
The IAM page listing the permissions for my project doesn't contain a member in the above format. The "App Engine default service account" is of the format: my-project-name#appspot.gserviceaccount.com. This service account has Cloud SQL Client and Editor roles.
While my queries are unsuccessful, after each attempt I note in the Logs Viewer:
7183 [Note] Access denied for user 'www-data'#'cloudsqlproxy~xxx.xxx.xx.xx' (using password: YES)
(IP address redacted). This is somewhat confusing as 'www-data' isn't a user I specified in my code.
The code used to connect:
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+pymysql://' + ':' + db_user + '#/' + db_name + '?unix_socket=/cloudsql/' + connection_name
Where have I gone wrong and how might I fix it?
This error is a mySQL error when trying to connect to a database with wrong credentials.
Please verify that the values you are using are the right ones.
If you dont rember the database username and password you can change it on the console by following the next steps which are also expalined here
Go to CloudSQL console
Select your database
Go to users
next to the user select click on the three dots
And select Change password
Type the new password and click OK
Mostly likely, you're building and testing your app locally where you're supplying your credentials with a username that has access to Cloud SQL. Upon build, unless you otherwise specify, a default username will be assigned to the app engine instance.
To fix this:
Head over to IAM & admin
Search for the app engine account --> ending with gae-api-prod.google.com.iam.gserviceaccount.com
Edit
Assign permission Cloud SQL Client
Let me know if this solves it for you!
We need to use an API to verify if a certain user exists as managed account (it means, that belongs to our Google Domain organization).
GSuite adminSDK performs that operation, however, it requires OAuth2 authentication, authorized by an authenticated user - https://developers.google.com/admin-sdk/reports/v1/guides/authorizing .
My question is, if there any way to use that API with service account, or any other methos to retrieve this information with service account, since it would be used in a Server-2-Server scenario.
Thanks,
Vasco
As you may know, Service Accounts don't belong to an individual end user, but to an application. An administrator of a G Suite domain, though, can authorize the Service Account to access user data, that is, to impersonate users in the domain. This is called domain-wide delegation.
To achieve this, go to the Admin console and follow the steps specified here.
Reference:
Delegating domain-wide authority to the service account
Reports API > Perform G Suite Domain-Wide Delegation of Authority
Since this is completely not obvious - the docs "hint" at this, but don't spell it out and I had trouble finding any specific examples that worked. They all kept returning this error:
Google.GoogleApiException: 'Google.Apis.Requests.RequestError
Not Authorized to access this resource/api [403]
Errors [
Message[Not Authorized to access this resource/api] Location[ - ] Reason[forbidden] Domain[global]
]
The basic issue that the service account MUST impersonate another user. It's mentioned in this link at the bottom, highlighted in blue:
https://developers.google.com/admin-sdk/directory/v1/guides/delegation
Only users with access to the Admin APIs can access the Admin SDK Directory API, therefore your service account needs to impersonate one of those users to access the Admin SDK Directory API. Additionally, the user must have logged in at least once and accepted the Google Workspace Terms of Service.
But it just wasn't clicking as to how I was supposed to do that - was this some setting hiding in one of the admin consoles? No - you pass this as part of your initial connection.
So just to put the instructions in one place and hopefully save someone else the same headache, these were the steps:
Created a project via https://console.developers.google.com
Searched for then enabled the Admin SDK API
Created a Service Account
Show domain-wide delegation / Enable G Suite Domain-wide Delegation
At this point I had a service account name, unique ID, Email, and Client ID
It generated a key file (json) that I downloaded.
Go to: https://admin.google.com
Security > API Controls.
Manage Domain Wide Delegation
Added entry using client ID from above, applied these two scopes - you can apply other scopes as needed.
https://www.googleapis.com/auth/admin.directory.user.readonly
https://www.googleapis.com/auth/admin.directory.group.member.readonly
Then I created a .NET Core console app and installed these NuGet packages:
Google.Apis
Google.Apis.Auth
Google.Apis.Admin.Directory.directory_v1
Here's an ugly proof of concept with everything working:
using System;
using System.IO;
using System.Net;
using Google.Apis.Admin.Directory.directory_v1;
using Google.Apis.Auth.OAuth2;
using Google.Apis.Services;
namespace GoogleDirectoryTest
{
class Program
{
static void Main(string[] args)
{
var dirService = GetDirectoryService(#"C:\tmp\google-cred-sample-12345abdef.json", "user-with-admin-permission-here#mydomainhere.com", "My App Name");
var list = dirService.Users.List();
list.Domain = "mydomainhere.com";
var users = list.Execute();
foreach (var user in users.UsersValue)
{
Console.WriteLine($"{user.Name.FullName}");
}
Console.ReadKey();
}
static DirectoryService GetDirectoryService(string keyfilepath, string impersonateAccount, string appName)
{
using (var stream = new FileStream(keyfilepath, FileMode.Open, FileAccess.Read))
{
var credentials = GoogleCredential.FromStream(stream).CreateWithUser(impersonateAccount);
if (credentials.IsCreateScopedRequired)
credentials = credentials.CreateScoped(new[] { DirectoryService.Scope.AdminDirectoryUserReadonly });
var service = new DirectoryService(new BaseClientService.Initializer()
{
HttpClientInitializer = credentials,
ApplicationName = appName,
});
return service;
}
}
Hopefully this saves someone else some headaches.
A service account is a special kind of account used by an application, rather than a person.
You can use a service account to access data or perform actions by the robot account itself, or to access data on behalf of Google Workspace or Cloud Identity users.
Prerequisites:
A Google Cloud Platform project
With the Admin SDK API enabled service account with domain-wide delegation.
A Google Workspace domain.
With account in that domain with administrator privileges.
Visual Studio 2013 or later
Step 1: Set up the Google Cloud Platform project
Create Google Cloud project
A Google Cloud project is required to use Google Workspace APIs and build Google Workspace add-ons or apps.
If you don't already have a Google Cloud project, refer to: How to create a Google Cloud project
Enable Google Workspace APIs
Before using Google APIs, you need to enable them in a Google Cloud project.
To Enable Google Workspace APIs refer to: How to Enable Google Workspace APIs
For this example you are enabling the the Admin SDK Directory API
with the data scope /auth/admin.directory.user.readonly.
Create Service Account with domain-wide delegation
To create service account refer to: How to create service account?
In the Domain wide delegation pane, select Manage Domain Wide Delegation.
Download Service Account private key (p12 format)
Download p12 file contains the private key for your Service Account.
Step 2: Set up the Google Workspace
Enable API access in the Google Workspace domain with
To enable API access in Google Workspace domain, refer to: how to enable API access
Delegating domain-wide authority to the service account
To call APIs on behalf of users in a Google Workspace organization, your service account needs to be granted domain-wide delegation of authority in the Google Workspace Admin console by a super administrator account
To delegating domain-wide authority in Google Workspace domain, refer to: How to Delegating domain-wide authority to the service account
Step 3: Prepare Visual Stodio project -
Create a new Visual C# Console Application (.NET Framework) project in Visual Studio.
Open the NuGet Package Manager Console, select the package source nuget.org, and run the following commands:
Install-Package Google.Apis.Auth
Install-Package Google.Apis.Admin.Directory.directory_v1
Step 4: Add code
Full example at GitHub
List top 10 users alias from Google Workspace Domain
/// <summary>
/// Example how to list all users from google workspace domain, using a service account (user impersonation).
/// </summary>
internal class Program {
static void Main(string[] args) {
// Scope for only retrieving users or user aliases.
string[] _scopes = {
"https://www.googleapis.com/auth/admin.directory.user.readonly"
};
var _paramters = new SACInitializeParameters(
// The service account ID (typically an e-mail address like: *#*iam.gserviceaccount.com)
serviceAccountId: "[Service Account ID]",
// The full path; name of a certificate file
x509CertificateFilePath: "[X509 Certificate File]",
// The email address of the user you trying to impersonate
impersonateEmail: "[User Email]",
// The scopes which indicate API access your application is requesting
scopes: _scopes);
using (var directoryService = DirectoryServiceFactory.CreateDirectoryService(_paramters)) {
// Retrieves a paginated list of either deleted users or all users in a domain.
var request = directoryService.Users.List();
// The unique ID for the customer's Google Workspace account
// the `my_customer` alias represent current identety account's
request.Customer = "my_customer";
request.MaxResults = 10;
var response = request.Execute();
foreach (var user in response.UsersValue) {
System.Console.WriteLine($"{user.Name.FullName}, {user.PrimaryEmail}, {user.Id}");
}
}
}
}
I'm trying to get the list of the intents in my Dialogflow agent using Dialogflow's V2 APIs but have been getting the following error:
PermissionDenied: 403 IAM permission 'dialogflow.intents.list' on 'projects/xxxx/agent' denied.
I adopted the following steps:
I created a new agent(with V2 APIs enabled) and a new service account for it.
I downloaded the JSON key and set my GOOGLE_APPLICATION_CREDENTIALS variable to its path.
Following is my code:
import dialogflow_v2 as dialogflow
os.environ["GOOGLE_APPLICATION_CREDENTIALS"]="/home/user/folder/service-account-key.json"
client=dialogflow.IntentsClient()
parent = client.project_agent_path('[PROJECT_ID]')
for element in client.list_intents(parent):
pass
I have made various agents and service accounts and even changed the role from Admin to Client but can't figure out any solution. I tried the following solution but didnt' work
Tried Solution: DialogFlow PermissionDenied: 403 IAM permission 'dialogflow.sessions.detectIntent'
There is no need for creating a new Agent. You can edit the existing agents IAM.
In Dialogflow's console, go to settings ⚙ > under the general tab, you'll see the project ID section with a Google Cloud link to open the Google Cloud console > Open Google Cloud.
In google cloud, go to IAM Admin > IAM under tab Members. Find the name of your agents and then click on edit.
Give admin permissions to the agent to give permissions to list intent.
The problem lies in the IAM section of GCP. Probably you are making a POST request with a role that does not have the necessary authorizations.
Look into your key.json file that contains the field "client_email"
Proceed to the IAM page and set the relevant role with that email to
a role that has posting capabilities. (e.g. Admin)
This solved my problem.
In Dialogflow's console, go to settings ⚙ > under the general tab, you'll see the project ID section with a Google Cloud link to open the Google Cloud console > Open Google Cloud.
(Optional) In the Cloud console, go to the menu icon > APIs & Services > Library. Select any APIs (if any) > Enable.
In Cloud Console > under the menu icon ☰ > APIs & Services > Credentials > Create Credentials > Service Account Key.
Under Create service account key, select New Service Account from the dropdown and enter a project name and for role choose Owner > Create.
JSON private key file will be downloaded to your local machine that you will need.
For Javascript:
In the index.js file you can do service account auth with JWT:
const serviceAccount = {}; // Starts with {"type": "service_account",...
// Set up Google Calendar Service account credentials
const serviceAccountAuth = new google.auth.JWT({
email: serviceAccount.client_email,
key: serviceAccount.private_key,
scopes: 'https://www.googleapis.com/auth/xxxxxxx'
});
For Python:
There's a Google Auth Python Library available via pip install google-auth and you can check out more here.
When you create the intentClient, use following:
key_file_path = "/home/user/folder/service-account-key.json";
client=dialogflow.IntentsClient({
keyFilename: key_file_path
})
Intents list
This error message is usually thrown when the application is not being authenticated correctly due to several reasons such as missing files, invalid credential paths, incorrect environment variables assignations, among other causes. Keep in mind that when you set an environment variable value in a session, it is reset every time the session is dropped.
Based on this, I recommend you to validate that the credential file and file path are being correctly assigned, as well as follow the Obtaining and providing service account credentials manually guide, in order to explicitly specify your service account file directly into your code; In this way, you will be able to set it permanently and verify if you are passing the service credentials correctly.
Passing the path to the service account key in code example:
def explicit():
from google.cloud import storage
# Explicitly use service account credentials by specifying the private key
# file.
storage_client = storage.Client.from_service_account_json('service_account.json')
# Make an authenticated API request
buckets = list(storage_client.list_buckets())
print(buckets)
Try also to create project in DialogFlow Console
https://dialogflow.cloud.google.com/
You need to create the following as environment variable
googleProjectID: "",
dialogFlowSessionID: "anything",
dialogFlowSessionLanguageCode: "en-US",
googleClientEmail: "",
googlePrivateKey:
I think you might have missed the Enable the API section in the documentation setup.
Here is that link:
https://cloud.google.com/dialogflow/cx/docs/quick/setup#api
After clicking the link, select the chatbot project you created and fill the necessary instructions given there.
The permissions that I have given for that project are Owner, and editor.
After this, try the code in this link:
https://cloud.google.com/dialogflow/es/docs/quick/api#detect_intent
You should get a response from your chatbot
Hope this helps!