PermissionDenied: 403 IAM permission 'dialogflow.intents.list' - google-cloud-platform

I'm trying to get the list of the intents in my Dialogflow agent using Dialogflow's V2 APIs but have been getting the following error:
PermissionDenied: 403 IAM permission 'dialogflow.intents.list' on 'projects/xxxx/agent' denied.
I adopted the following steps:
I created a new agent(with V2 APIs enabled) and a new service account for it.
I downloaded the JSON key and set my GOOGLE_APPLICATION_CREDENTIALS variable to its path.
Following is my code:
import dialogflow_v2 as dialogflow
os.environ["GOOGLE_APPLICATION_CREDENTIALS"]="/home/user/folder/service-account-key.json"
client=dialogflow.IntentsClient()
parent = client.project_agent_path('[PROJECT_ID]')
for element in client.list_intents(parent):
pass
I have made various agents and service accounts and even changed the role from Admin to Client but can't figure out any solution. I tried the following solution but didnt' work
Tried Solution: DialogFlow PermissionDenied: 403 IAM permission 'dialogflow.sessions.detectIntent'

There is no need for creating a new Agent. You can edit the existing agents IAM.
In Dialogflow's console, go to settings ⚙ > under the general tab, you'll see the project ID section with a Google Cloud link to open the Google Cloud console > Open Google Cloud.
In google cloud, go to IAM Admin > IAM under tab Members. Find the name of your agents and then click on edit.
Give admin permissions to the agent to give permissions to list intent.

The problem lies in the IAM section of GCP. Probably you are making a POST request with a role that does not have the necessary authorizations.
Look into your key.json file that contains the field "client_email"
Proceed to the IAM page and set the relevant role with that email to
a role that has posting capabilities. (e.g. Admin)
This solved my problem.

In Dialogflow's console, go to settings ⚙ > under the general tab, you'll see the project ID section with a Google Cloud link to open the Google Cloud console > Open Google Cloud.
(Optional) In the Cloud console, go to the menu icon > APIs & Services > Library. Select any APIs (if any) > Enable.
In Cloud Console > under the menu icon ☰ > APIs & Services > Credentials > Create Credentials > Service Account Key.
Under Create service account key, select New Service Account from the dropdown and enter a project name and for role choose Owner > Create.
JSON private key file will be downloaded to your local machine that you will need.
For Javascript:
In the index.js file you can do service account auth with JWT:
const serviceAccount = {}; // Starts with {"type": "service_account",...
// Set up Google Calendar Service account credentials
const serviceAccountAuth = new google.auth.JWT({
email: serviceAccount.client_email,
key: serviceAccount.private_key,
scopes: 'https://www.googleapis.com/auth/xxxxxxx'
});
For Python:
There's a Google Auth Python Library available via pip install google-auth and you can check out more here.

When you create the intentClient, use following:
key_file_path = "/home/user/folder/service-account-key.json";
client=dialogflow.IntentsClient({
keyFilename: key_file_path
})
Intents list

This error message is usually thrown when the application is not being authenticated correctly due to several reasons such as missing files, invalid credential paths, incorrect environment variables assignations, among other causes. Keep in mind that when you set an environment variable value in a session, it is reset every time the session is dropped.
Based on this, I recommend you to validate that the credential file and file path are being correctly assigned, as well as follow the Obtaining and providing service account credentials manually guide, in order to explicitly specify your service account file directly into your code; In this way, you will be able to set it permanently and verify if you are passing the service credentials correctly.
Passing the path to the service account key in code example:
def explicit():
from google.cloud import storage
# Explicitly use service account credentials by specifying the private key
# file.
storage_client = storage.Client.from_service_account_json('service_account.json')
# Make an authenticated API request
buckets = list(storage_client.list_buckets())
print(buckets)

Try also to create project in DialogFlow Console
https://dialogflow.cloud.google.com/

You need to create the following as environment variable
googleProjectID: "",
dialogFlowSessionID: "anything",
dialogFlowSessionLanguageCode: "en-US",
googleClientEmail: "",
googlePrivateKey:

I think you might have missed the Enable the API section in the documentation setup.
Here is that link:
https://cloud.google.com/dialogflow/cx/docs/quick/setup#api
After clicking the link, select the chatbot project you created and fill the necessary instructions given there.
The permissions that I have given for that project are Owner, and editor.
After this, try the code in this link:
https://cloud.google.com/dialogflow/es/docs/quick/api#detect_intent
You should get a response from your chatbot
Hope this helps!

Related

Upload GCS File to Google Drive (Airflow)

I am using airflow.providers.google.suite.transfers.gcs_to_gdrive.GCSToGoogleDriveOperator
to upload a file from GCS to Google Drive.
Getting 403 Forbidden with reason "insufficientPermissions"
This is the logging. Not sure where and what the issue is. any help is highly appreciated !!!
{gcs_to_gdrive.py:151} INFO - Executing copy of gs://adwords_data/conversion_data.csv to gdrive://google_adwords/
{gcs.py:328} INFO - File downloaded to /tmp/tmpzmk4n2z9
{http.py:126} WARNING - Encountered 403 Forbidden with reason "insufficientPermissions"
Code
from airflow.providers.google.suite.transfers.gcs_to_gdrive import GCSToGoogleDriveOperator
copy_google_adwords_from_gcs_to_google_drive = GCSToGoogleDriveOperator(
task_id="copy_google_adwords_from_gcs_to_google_drive",
source_bucket="{}".format(gcs_to_gdrive_bucket),
source_object="conversion_data.csv",
destination_object="adwords_data/",
gcp_conn_id='google_cloud_default',
dag=dag
)
In the gcp_conn_id = google_cloud_default. I have added the scope = https://www.googleapis.com/auth/drive
In your airflow connection, in the scopes field, try adding:
https://www.googleapis.com/auth/drive, https://www.googleapis.com/auth/cloud-platform
You'll also need to ensure the service account has the correct roles assigned (as well as the drive being shared w/ the Service Account's email)
Some of the roles you can add to the service account by going to IAM > click edit on the service account, and add the roles

Integrate AWS Cognito with Google Workspace using SAML integration

I have some applications served to my company users on EKS (i.e., Jenkins). In company we use Google Workspaces (GSuite) for email and stuff. So I want to allow users to login with Google creds to those applications I serve. I figured out I could use Cognito to achieve it but I cannot connect those and flow end with Google showing 403. Error: app_not_configured_for_user. In their documentation I can find:
Verify that the value in the saml:Issuer tag in the SAMLRequest matches the Entity ID value configured in the SAML Service Provider Details section in the Admin console. This value is case-sensitive.
but how do I debug it? I do not see a logs from neither AWS and Google sides :/
I think I followed all possible guides and I cannot find what I'm doing wrong. I found that Google has this page but they do not provide exact scenario for AWS Cognito. Anyways all of those are very similar so I guess I shouldn't have problems, but I do have.
What I did:
In Google Admin (one for workspaces) I created "Web and mobile app" of SAML type
I downloaded metadata file
In AWS Cognito console I created User Pool
I created IdP provider and uploaded metadata file there
I created application client
Using those values I filled fields ACS URL and Entity ID in Google Admin using values:
ACS URL: https://my-domain-i-just-created.auth.us-east-1.amazoncognito.com/saml2/idpresponse
Entity ID: urn:amazon:cognito:sp:us-east-1_myPoolId
I also selected Name ID format to be Persisted
In attribute mapping I mapped email value to http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress.
In AWS Cognito I enabled HostedUI and also created mapping of http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress to email field.
And now when I click View Hosted UI in AWS console it will redirect me to Google authentication and after it directly to before mentioned 403 app_not_configured_for_user page.
I tied it 3 times with slightly different configurations of mapping, signed responses, etc. but nothing gets me past that error.
Anyone tried to integrate it?
How to troubleshoot the 403 app_not_configured_for_user error related to SAML apps from the Google Workspace Admin console
The first thing you need to do is to grab a HAR file recording the whole login process and find the SAML request. Steps can be found here.
Once you get the file you can open it using that tool and search for SAMLreq at the top right (see image).
After that you will get a list of values containing information. You will have to check one by one until you find the one that has the SAML request in the request tab (see example below).
Once you get the value from the SAML request, copy it and you can use this tool to do a SAML decode and find the entity ID. You can use Ctrl + F and search for saml:Issuer to find the value faster. If the value does not match, then you know you have an error and you will need to contact the support team from the app to see which value is the correct one.
In case the value matches I would recommend opening a ticket to check with Google.

Google sheets API v4 permissions PERMISSION_DENIED error using POSTMAN [duplicate]

I've generated a server key in the API Manager and attempted to execute the following on my Mac:
curl 'https://sheets.googleapis.com/v4/spreadsheets/MySheetID?ranges=A1:B5&key=TheServerKeyIGeneratedInAPIManager'
But this is what it returns:
{
"error": {
"code": 403,
"message": "The caller does not have permission",
"status": "PERMISSION_DENIED"
}
}
What am I doing wrong here?
To solve this issue, try to:
Create a service account: https://console.developers.google.com/iam-admin/serviceaccounts/
In options, create a key: this key is your usual client_secret.json - use it the same way
Make the role owner for the service account (Member name = service account ID = service account email ex: thomasapp#appname-201813.iam.gserviceaccount.com
Copy the email address of your service account = service account ID
Simply go in your browser to the Google sheet you want to interact with
Go to SHARE on the top right of your screen
Go to advanced settings and share it with an email address of your service account ex: thomasapp#appname-201813.iam.gserviceaccount.com
I know it is a little late to answer but for other people struggling with the same issue.
Just change the permission of the sheet to public on your drive so it can be accessed without authentication via API calls.
To change access:
Open sheet in google drive
On top right corner, click share
On bottom of prompt window, click advanced
Change permission to public or people with link (no signin required)
Send API request to fetch data from sheets without authentication.
Note: if the sheet contains sensitive data then it is not safe to make it public and rather do it with Authenticated access.
Make sure to pay attention to #KishanPatel's comment:
Also, you can share this sheet with specific email Ex. your service
account (project) email. "client_email":
"XXXXX#northern-gasket-XXXX.iam.gserviceaccount.com", This will allow
to access sheet by your script.
Visual Simplification of the Answers:
Option 1 - Turn the file into public (if sheets the sheet contains sensitive data)
Option 2 - Share file with Service Account Email (IAM & Admin -> Service Accounts -> Details -> Email)
The easiest way is to fix using gcloud cli. More docs here https://cloud.google.com/pubsub/docs/quickstart-cli#before-you-begin
install gcloud
sudo apt-get install google-cloud-sdk
then call
gcloud init
then check your active project and credentials
gcloud config configurations list
If it is not ok, make sure you are authenticated with the correct account:
gcloud auth list
* account 1
account 2
Change to the project's account if not:
gcloud config set account `ACCOUNT`
Depending on the account, the project list will be different:
gcloud projects list
- project 1
- project 2...
Switch to intended project:
gcloud config set project `PROJECT NAME`
Then Create Application Default Credentials with gcloud auth application-default login, and then google-cloud will automatically detect such credentials.
My 10 cents... A simple example to read the sheet using Java.
private Credential getCredentials() throws IOException {
final InputStream accessKey = new ByteArrayInputStream("<credential json>");
final GoogleCredential credential = GoogleCredential.fromStream(accessKey)
.createScoped(Collections.singleton(SheetsScopes.SPREADSHEETS_READONLY));
return credential;
}
private HttpTransport httpTransport() {
try {
return GoogleNetHttpTransport.newTrustedTransport();
} catch (GeneralSecurityException | IOException e) {
throw new SpreadSheetServiceException(e);
}
}
Sheets service = new Sheets.Builder(httpTransport(), JSON_FACTORY, getCredentials())
.setApplicationName("app-name")
.build();
ValueRange response = service.spreadsheets().values()
.get("<spread_sheet_id>", "A1:A")
.execute();
In my case, solving this problem turned out to be trivial. You just have to:
Enter the google sheet that we want to remotely edit.
In the upper right corner, set - anyone who has the link can enter
Most importantly - on the right side you need to set permissions for people who have the link as 'editor'
if you still do not have permission, it means that you have to go to the website:
https://console.developers.google.com/iam-admin/iam/ then select your project, then select "Service accounts" and create a new one as role "owner" or" editor" for the project for example (or use one that already exists and click "create new key")
The "key" is a json file that will be downloaded when you create the account (or use "create new key" there).

Connecting App Engine to Cloud SQL Access Denied

I'm trying to get my Flask App Engine project to read data stored in Cloud SQL MySQL 5.7 database. Something has gone wrong as all I've gotten are pymysql.err.OperationalError. I've been following the instructions here: https://cloud.google.com/sql/docs/mysql/connect-app-engine.
The Cloud SQL Admin API is enabled.
According to the linked document:
App Engine uses a service account to authorize your connections to Cloud SQL. This service account must have the correct IAM permissions to successfully connect. Unless otherwise configured, the default service account is in the format service-PROJECT_NUMBER#gae-api-prod.google.com.iam.gserviceaccount.com.
The IAM page listing the permissions for my project doesn't contain a member in the above format. The "App Engine default service account" is of the format: my-project-name#appspot.gserviceaccount.com. This service account has Cloud SQL Client and Editor roles.
While my queries are unsuccessful, after each attempt I note in the Logs Viewer:
7183 [Note] Access denied for user 'www-data'#'cloudsqlproxy~xxx.xxx.xx.xx' (using password: YES)
(IP address redacted). This is somewhat confusing as 'www-data' isn't a user I specified in my code.
The code used to connect:
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+pymysql://' + ':' + db_user + '#/' + db_name + '?unix_socket=/cloudsql/' + connection_name
Where have I gone wrong and how might I fix it?
This error is a mySQL error when trying to connect to a database with wrong credentials.
Please verify that the values you are using are the right ones.
If you dont rember the database username and password you can change it on the console by following the next steps which are also expalined here
Go to CloudSQL console
Select your database
Go to users
next to the user select click on the three dots
And select Change password
Type the new password and click OK
Mostly likely, you're building and testing your app locally where you're supplying your credentials with a username that has access to Cloud SQL. Upon build, unless you otherwise specify, a default username will be assigned to the app engine instance.
To fix this:
Head over to IAM & admin
Search for the app engine account --> ending with gae-api-prod.google.com.iam.gserviceaccount.com
Edit
Assign permission Cloud SQL Client
Let me know if this solves it for you!

Access Denied on Windows for AWS Credentials file

No matter what I try it seems my web service cannot access my .aws/credentials file.
I always get this error:
System.UnauthorizedAccessException: Access to the path '{PATH}' is denied.
Here is what I have tried:
Move the path from the default directory to the website root
Change the website app pool to run as my user account
Given Everyone full control of the folder and the file
Verify that when I put the same key and secret into the web.config the call works
Tried removing the region from the config
Tried removing the path from the config
Here is my config (note if I don't provide the path, even when in the default location, it says no credentials file was found)
<add key="AWSProfileName" value="default" />
<add key="AWSRegion" value="us-east-1"/>
<add key="AWSProfilesLocation" value="{PATH}" />
In the AWS toolkit I have a `default' profile setup as well that has rights but that does not help this work.
I have even tried the legancy format called out in the AWS docs. What am I missing? It seems I have followed everything AWS calls out in their docs.
I am using Castle Windsor DI so could that be getting in the way?
container.Register(
Component.For<IAmazonDynamoDB>()
.ImplementedBy<AmazonDynamoDBClient>()
.DependsOn(Dependency.OnValue<RegionEndpoint>(RegionEndpoint.USEast1))
.LifestylePerWebRequest());
container.Register(
Component.For<IDynamoDBContext>()
.ImplementedBy<DynamoDBContext>()
.DependsOn(Dependency.OnComponent<IAmazonDynamoDB, AmazonDynamoDBClient>())
.DependsOn(Dependency.OnValue<DynamoDBContextConfig>(
new DynamoDBContextConfig
{
TableNamePrefix = configurationManager.GetRequiredAppSetting<string>(Constants.Web.AppSettings.AwsDynamoDbPrefix),
Conversion = DynamoDBEntryConversion.V2
}))
.LifestylePerWebRequest());
The problem that you have is that the path ~\.aws\credentials is only defined when logged in as a user.
A Windows services such as IIS is not logged in as the user that created the credentials file. Therefore the the path is not accessible to the Windows service. Actually the service does not know what user to look into. For example if your user name is john, the path would be c:\users\john\.aws\credentials. The Windows service does not know about your identity.
Note: I believe - but I am not 100% sure - is that a windows service will look in c:\.aws for credentials. I have used this path in the past but I cannot find Amazon reference documentation to support this. I no longer store credentials on my EC2 instances, so I am out of touch on the location c:\.aws.
You have a number of choices. Create the credentials as usual. Then create a directory outside of your IIS installation and setup such as c:\.aws. Copy ~\.aws to c:\.aws. Then specify the full path in your programs.
A much better and more secure method, if you are running your services on AWS, is to use IAM Role. Create a role with the desired permissions and attach the role to your EC2 instance. All AWS SDKs and Tools know how to find the credentials from AWS Metadata.
There are many more methods such as EC2 Parameter Store. Storing credentials on your instances or inside your program is not a good idea.
[Edit after thinking more about the error message]
You may have an issue where IIS does not have access rights to the location where the credentials are stored.
Open Windows Explorer and locate the folder for your credentials file. Right click this folder, select Properties and click the Security tab. From here, choose Edit then Add. The following users must be added and given at least READ permissions: IUSR & IIS_IUSRS. You may need to add "LIST FOLDER CONTENTS".