I have parameterized views created in snowflake. I need to set parameters to query the parameterized views.
Can you please help to set snowflake session variable through powerbi before running select query on view.
Set dat_parameter='2022-01-01'
Select * from parameterized_view
Power bi is not accepting set statement
By setting enablefolding=false as shown below in power bi and setting multi+statement_count = 0 in snowflake. I am able to extract the data from parameterized view in snowflake.
power bi:
= Value.NativeQuery(Snowflake.Databases("XXXXXXXXX.snowflakecomputing.com","compute_wh"){[Name="TESTDB"]}[Data],
"set abc=60001;select * from testdb.public.param_view", null, [EnableFolding=false])
Snowflake:
alter account set MULTI_STATEMENT_COUNT = 0;
parameterized view creation:
create or replace view param_view AS
select * from SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.CUSTOMER where C_CUSTKEY = $abc;
Related
I need to filter my Power BI report by the App IDs associated with the current user (using the USERPRINCIPALNAME function). So I have three tables in my model, DimApp, DimUser, and FactRegisters, where a User_Id may be related to 1 or more App_Ids in my Fact table.
DimApp table
DimUser table
FactRegister table
As you can see in FactRegisters table there are two App_Ids (3 and 1) for User_Id 201. The following is the DAX rule defined in App_Id column from DimApp table to filter the data:
VAR userId =
LOOKUPVALUE (
DimUser[User_Id],
DimUser[Email], USERPRINCIPALNAME()
)
VAR app =
LOOKUPVALUE (
FactRegisters[Application_Id],
FactRegisters[User_Id], userId
)
RETURN DimApplication[Application_Id] IN {app}
Verifying the DAX expression doesn't return an error, however, when I choose to "View as" that role I'm not able to see the data in the visuals. The error states: "Couldn't load the data for this visual. An error was encountered during the evaluation of the row-level security expression defined in table DimApp. A table of multiple values was supplied where a single value was expected."
Cannot display the visual viewing as role
However, when a single App_Id is associated with the User_Id, I'm able to visualize the data on the report visuals using the same DAX rule. Here is how FactRegisters table looks like when User_Id 201 has a single App_Id (3) associated:
FactRegisters table when User_Id with single App_Id
User_Id with a single App_Id visual
Now I'm I able to visualize data in the report. This is not a suitable case scenario as a User_Id can have many App_Ids.
I also tried the following static DAX rule in my App_Id column from DimApp just to test and pass multiple values to that column, and I succeed in visualizing data for multiple App_Ids:
DimApplication[Application_Id] IN {1,3}
Static RLS with multiple values by App_Id column
But this is not the goal (it's not dynamic). The goal is to visualize the data from all the Apps associated with the current user. Is it possible? Can't I pass more than one value to a column while filtering in RLS?
[Application_Id] IN
CALCULATETABLE(
VALUES('FactRegister'[Application_Id]),
FILTER ('FactRegister',
'FactRegister'[User_Id] = LOOKUPVALUE (DimUser[User_Id], DimUser[Email],USERPRINCIPALNAME())
)
)
I have a gcp based environment. I use standard SQL scripting in gcp BigQuery and federated query to cloudsql MySql. Federated query selects data from cloudsql mysql database. I need to select data from cloudsql mysql database based on condition that depends on data in BigQuery. I use variables in standard sql scriping in gcp bigquery to store the value that I select from bigquery. I want to value of this variable in the where clause of mysql query. See following example where I select a date from BigQuery and store it in a variable "BQ_LAST_DATETIME".
DECLARE BQ_LAST_DATETIME DATETIME
SET BQ_LAST_DATETIME = (select max(date_created) from bq_my_dataset.bq_my_table);
Since I am using bigquery federated query to read data out of cloudsql database (https://cloud.google.com/bigquery/docs/cloud-sql-federated-queries) as shown below and I want to use value that I stored in the variable "BQ_LAST_DATETIME" in the mysql query where clause
SELECT * FROM EXTERNAL_QUERY("my-gcp-project.my-region.my-connection2-cloudsql", "select * from mysqlschema.mysql_table where where date_created = #BQ_LAST_DATETIME;" );
Please note that in above query I have used "#BQ_LAST_DATETIME" as a placeholder to show what I want to achieve. I am not sure if I can directly use bigquery scripting variable as query parameter in the "external" query part of federated query.
Any suggestions on how to achieve parametrization of external queries in federated query, or if you know how I could achieve effect similar to what my intent is?
I actually tried following as depicted . I used bigquery scripting variable as query parameter in the "external" query part of federated query. only nuance here is that since the I was dealing with dates I performed a cast and also since the date variable actually is treated as a string I formatted it back to date using mysql STR_TO_DATE as follows
DECLARE BQ_LAST_DATETIME DATETIME
SET BQ_LAST_DATETIME = (select max(date_created) from bq_my_dataset.bq_my_table);
SET BQ_LAST_DATE= CAST(BQ_LAST_DATETIME AS DATE);
SELECT * FROM EXTERNAL_QUERY("my-gcp-project.my-region.my-connection2-cloudsql", "select * from mysqlschema.mysql_table where where date_created = STR_TO_DATE(#BQ_LAST_DATE,'%Y-%m-%d') ;" );
While this query is accepted by parser it is NOT giving expected result.
Basically the value of the variable #BQ_LAST_DATE does not seem to get to MySQL query as expected.
Does anyone know what am I missing ?
Thanks a lot for your help
You can try EXECUTE IMMEDIATE:
DECLARE BQ_LAST_DATETIME STRING;
DECLARE DSQL STRING;
SET BQ_LAST_DATETIME = 'SELECT max(date_created) from bq_my_dataset.bq_my_table';
SET DSQL = '"select * from mysqlschema.mysql_table where date_created = (' || BQ_LAST_DATETIME || ')"';
EXECUTE IMMEDIATE 'SELECT * FROM EXTERNAL_QUERY("my-gcp-project.my-region.my-connection2-cloudsql",' || DSQL || ');'
I have created the following dataframes in databricks:
salebycountry = spark.read.csv("/FileStore/tables/SalesByCountry.csv",inferSchema=True,header=True)
stock = spark.read.csv("/FileStore/tables/Data_Stock.csv",inferSchema=True,header=True)
model = spark.read.csv("/FileStore/tables/Data_Model.csv",inferSchema=True,header=True)
salesdetails = spark.read.csv("/FileStore/tables/Data_SalesDetails.csv",inferSchema=True,header=True)
make = spark.read.csv("/FileStore/tables/Data_Make.csv",inferSchema=True,header=True)
sales = spark.read.csv("/FileStore/tables/Sales.csv",inferSchema=True,header=True)
However, when I try to view / load the data into Power BI with Power BI Desktop I am only able to see the dataframe if I issue the command .write.saveAsTable(). For example, if I want to see the dataframe called 'model', I need to write the following code model.write.saveAsTable('Model').
I've never had to do that in the past to view dataframes. I'm wondering if its because in this case I uploaded the data(csv) into databricks as opposed to ingesting the data via SQL server? But I'm not sure.
I want to create a chart from a join sql query between 2 tables in superset.
for example , I go to SQL Lab and execute this query :
select film, count("film") from rental r, payment p where r.rental_id=p.rental_id group by("film") order by count("film") limit 20;
This returns me a result but how to insert in a chart?
How to create chart from SQL query ?
In order to visualize the results from a query executed in SQL Lab, you first need to click on Explore (underneath the Results tab).
Once you are in exploration mode, you can change the "Visualization Type", under "Datasource & Chart Type".
I'm looking for a sample code which get data from SQL Server and push this to PowerBI in real time, This is basically using the Push Dataset option.
I am not sure how to Push the datas from SQL
Thanks
Why not creating a custom streaming dataset and 'pushing' your sql data directly. In this case you may use either Power apps (create a flow and a trigger on insert) or simply right some code to push your data in a form of a post request.
For instance you have your sql table containing a value you want to push. Thus the steps should the following:
Create a dashboard
Add tile
Choose 'Custom Streaming Dataset' as a source
Define the data colums to be pushed (for instance train_number and departure_time)
Copy the API
From your code (Python for example) get the data, convert it to json and publish
Go back to power bi, add a tile from newly created streaming dataset and chose the visual type. Important: the visuals are quite limited
Here is a sample code in python:
def data_generation(counter=None):
# get your SQL data and save it into 2 variables (row by row)
return [train_number, departure_time]
while True:
data_raw = []
# simple counter increment
counter += 1
for i in range(1):
row = data_generation(counter)
data_raw.append(row)
# set the header record
HEADER = ["train_number", "departure_time"]
# generate a temp data frame to convert it to json
data_df = pd.DataFrame(data_raw, columns=HEADER)
# prepare date for post request (to be sent to Power BI)
data_json = bytes(data_df.to_json(orient='records'), encoding='utf-8')
# Post the data on the Power BI API
req = requests.post(PowerBI_REST_API_URL, data_json)
print("Data posted in Power BI API")
print(data_json)
# wait 5 seconds
time.sleep(5)
Microsoft published similar walk-through. It has to be slightly expanded with SQL Server calls though:
Push data into a Power BI dataset
---> Create Dataset
You can't 'push' data from SQL, but you can use DirectQuery instead of Import. Then your data will always be actual.
Just connect to a SQL Server, and choose for 'Direct Query' and you'll be ready to go.
Edit:
With #Alexander Volok, of course, with an application and/or API calls you can push data into Power BI. My bad.
You Can push the data by using power shell which where you need to add the your api link and you have to put your sql connection string and you and you ca fire a query to same data set by declaring it into code you can refer the below code which will help you to understand how to push the data into your data set once you run your power shell script then data will be pushed to power bi data set and you can see your live
$SqlServer = ''; #your server name
$SqlDatabase = ''; #your database name
$uid ="" #User id
$pwd = "*****" # your password
$SqlConnectionString = 'Data Source={0};Initial Catalog={1};Integrated Security=SSPI;uid=$uid;Password=$pwd' -f $SqlServer, $SqlDatabase;
$SqlQuery = "SELECT * FROM abc;";
$SqlCommand = New-Object System.Data.SqlClient.SqlCommand;
$SqlCommand.CommandText = $SqlQuery;
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection -ArgumentList $SqlConnectionString;
$SqlCommand.Connection = $SqlConnection;
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCommand
$SqlConnection.Open();
$SqlDataReader = $SqlCommand.ExecuteReader();
##you would find your own endpoint in the Power BI service
$endpoint = "" ## add your api link middle of endpoint ""
#Fetch data from your table and write out to files
while ($SqlDataReader.Read()) {
$payload =
#{
"Date" =$SqlDataReader['Date']
"First Name" =$SqlDataReader['Name']
"Production" =$SqlDataReader['prdt']
}
Invoke-RestMethod -Method Post -Uri "$endpoint" -Body (ConvertTo-Json #($payload))
}
$SqlConnection.Close();
$SqlConnection.Dispose();
## every time you run script data will automaticaly pushed from sql server to your power bi report
e streaming chart