CRMint - Import audiences from BigQuery to Google Analytics - google-cloud-platform

I am not sure what's the easiest way to import audiences from BigQuery to Google Analytics. I have a BigQuery table called dataset_a.georgia with 2 columns fullVisitorId (string) and predictions (string).
Based on CRMint documentation, there is a worker called GAAudiencesUpdater which is doing what I am looking for.
However, one of the required parameters is a GA audience JSON template. I have a hard time to understand how exactly I should write this JSON as the basic JSON example is pretty long and hard to understand. I just have 2 columns to import, the fullVisitorId is already a default variable in Google Analytics while I just created a custom dimension called predictions with the index 183 and user scope.
I talked to a senior developer who had a bit of experience with CRMint and he suggested me to not use the GAAudiencesUpdater because of how hard it is to write the GA audience JSON template. He suggested me to import my BigQuery data into a CSV, store that CSV into Google Cloud Storage before to import that CSV to Google Analytics with the worker to GADataImporter.
Any suggestions?

They updated the version. You don't have to import JSON now.

You can find here an example
GAAudiencesUpdater
Here's the basic Google Analytics JSON template example:
{
"audienceType": "STATE_BASED",
"linkedAdAccounts": [
{
"linkedAccountId": "{% AW_ACCOUNT_ID %}",
"type": "ADWORDS_LINKS"
}
],
"linkedViews": [
{% GA_VIEW_ID %}
],
"name": "CRMint Tier GA %(tier)i",
"stateBasedAudienceDefinition": {
"includeConditions": {
"daysToLookBack": 7,
"segment": "users::condition::ga:dimension{% CD_TIER %}=#%(code)s",
"isSmartList": false,
"membershipDurationDays": %(duration)i
},
"excludeConditions": {
"segment": "users::condition::ga:dimension{% CD_TIER %}==dur93j#%(tier)ihg#2d6",
"exclusionDuration": "TEMPORARY"
}
}
}

Related

Get proper usernames to populate on Superset with Azure SSO instead of ID string

I've finally gotten Azure Single Sign-On (SSO) connected to Apache Superset running via docker-compose, following the Flask docs. Users in my company's Azure group can create and access Superset accounts by logging in with Azure and they are assigned roles based on their identity. This is good.
The usernames they get assigned, however, are long Azure ID strings. These are undesirable in displays. Here's what my account looks like on the List Users screen and on my profile:
How can I modify either my Azure application SSO setup or my Superset config to have Superset populate usernames like SFirke for the account usernames, instead of values like 3ee660ff-a274 ... ?
The security part of my config.py looks like this, almost identical to the Flask template:
OAUTH_PROVIDERS = [
{
"name": "azure",
"icon": "fa-windows",
"token_key": "access_token",
"remote_app": {
"client_id": "CLIENT_ID",
"client_secret": "CLIENT_SECRET",
"api_base_url": "https://login.microsoftonline.com/TENANT_ID/oauth2",
"client_kwargs": {
"scope": "User.read name preferred_username email profile upn groups",
"resource": "RESOURCE_ID",
},
"request_token_url": None,
"access_token_url": "https://login.microsoftonline.com/TENANT_ID/oauth2/token",
"authorize_url": "https://login.microsoftonline.com/TENANT_ID/oauth2/authorize",
},
},
]
EDIT: Looks like the way to go is writing a custom userinfo retrieval method, there's a template on the Flask page linked above and an example used for Superset in this Github comment. I think I would use a line like "id": me["preferred_username"] or "id": me["upn"], based on the field names in the Microsoft docs.
But Microsoft notes that this value can change over time and should not be used for authorization changes. Since the oid value is immutable, and it is hardly visible to the typical user, I plan to just stick to it.

aws amplify datastore syncing the whole database

In my DynamoDb I have about 200k datapoints, there will be more in the future. When I logout my local datastorage gets cleared. When I log in, datastore starts to sync it with the cloud. The problem is that the syncing is taking really long for over 200k datapoints. The datapoints are sensorik data that its displayed on a chart.
My idea is to fetch only the data directly from the database which I need without bloating up my entire local storage.
Is there a way to fetch the data what I need without saving it into the offline storage? I was thinking to rather use AWS timeseries for my chart data.
SyncExpression Configuration is required for fetch the specific data based on your need.
DOC: https://docs.amplify.aws/lib/datastore/sync/q/platform/js/
import { DataStore, syncExpression } from 'aws-amplify';
import { Post, Comment } from './models';
DataStore.configure({
syncExpressions: [
syncExpression(Post, () => {
return post => post.rating.gt(5);
}),
syncExpression(Comment, () => {
return comment => comment.status.eq('active');
}),
]
});

How to import knowledge base through api?

https://cloud.google.com/dialogflow/es/docs/reference/rest/v2beta1/projects.knowledgeBases.documents/import
Consider I'm having an csv file to be imported in a cloud storage, How exactly do I execute this above API request and import the knowledge base qna's, I've added the documentation link above, I'm getting the below error too
Change the parent to projects/your-project-id/knowledgeBases/xxxxxx and import should accept it.
But I suggest to use projects.knowledgeBases.documents.create if you are planning to create a knowledge base from scratch using a CSV file. See sample request via projects.knowledgeBases.documents.create:
parent: projects/your-project-id/knowledgeBases/xxxxx
importGcsCustomMetadata: false
Request Body:
{
"contentUri": "gs://my-bucket/faq.csv",
"displayName": "test_csv",
"knowledgeTypes": [
"FAQ"
],
"mimeType": "text/csv"
}
Returns HTTP 200:
{
"name": "projects/your-project-id/locations/us/operations/document-create-20210829-21261630297603-6127fbb9-0000-21dc-bec9-240588717654"
}
Created knowledge base in Dialogflow console:

Creating a dataset with Power BI's REST API and associate it with a custom connector

I would like to know if it is currently possible to create a dataset in Power BI service, using the Power BI REST API, so that the dataset is associated with a data source whose type corresponds to a certified custom connector in Power BI.
In my scenario I have a Power BI custom connector that allows me to connect to and access data at a server in DirectQuery mode. I want to create a dataset in Power BI service through the Power BI REST API and connect it to a data source in my gateway. This data source has "Extension" as "datasourceType" because I use the custom connector.
Data source created with the Power BI REST API:
{
"id": "7077dcea-6874-42ec-86df-40d3f4af7c50",
"gatewayId": "52bbd773-b6c3-4b86-a6f0-e8a15551bc53",
"datasourceType": "Extension",
"connectionDetails": "{\"extensionDataSourceKind\":\"DenodoForPowerBI\",\"extensionDataSourcePath\":\"{\\\"DSN\\\":\\\"DenodoODBC\\\"}\"}",
"credentialType": "Basic",
"credentialDetails": {
"useEndUserOAuth2Credentials": false
},
"datasourceName": "Testing_REST_API_02"
}
Now I want to create a dataset bound to this data source. According to the documentation (https://learn.microsoft.com/en-us/rest/api/power-bi/pushdatasets/datasets_postdataset), it seems that the body of the request should be something like this:
{
"name": "datasettest",
"tables": [],
"datasources": [
{
"datasourceType": "Extension",
"gatewayId": "52bbd773-b6c3-4b86-a6f0-e8a15551bc53",
"datasourceId": "7077dcea-6874-42ec-86df-40d3f4af7c50"
}
]
}
This POST requests throws an error:
{
"error": {
"code": "InvalidRequest",
"message": "TOM deserialization failed for database sobe_wowvirtualserver|c7da28ca-2b93-438e-9341-ddd9ceba4c61, Exception: <pi>Unrecognized JSON property: datasources. Check path 'datasources', line 1, position 67.</pi>"
}
}
I do not know if this type of connector needs any specification that I am not taking into account or if this functionality is not supported.
I will be grateful for any help you can provide.

Amazon API Gateway swagger importer tool does not import minItems feild from swagger

I am trying the api gateway validation example from here https://github.com/rpgreen/apigateway-validation-demo . I observed that from the given swagger.json file, minItems is not imported into the models which got created during the swagger import.
"CreateOrders": {
"title": "Create Orders Schema",
"type": "array",
"minItems" : 1,
"items": {
"type": "object",
"$ref" : "#/definitions/Order"
}
}
Because of this when you give an empty array [ ] as input, instead of throwing an error about minimum items in an array, the api responds with a message 'created orders successfully'.
When I manually add the same from the API gateway console UI, it seems to work as expected. Am i missing something or this is a bug in the importer?
This is a known issue with the Swagger import feature of API Gateway.
From http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-known-issues.html
The maxItems and minItems tags are not included in simple request validation. To work around this, update the model after import before doing validation.