Need to know whether Power BI supports parquet format as source from adls gen1.
Planning to use adls gen1 or databricks delta lake(supports parquet format only) as source to get data into power bi.Kindly suggest or please share any documentation link to get an idea related to it.
Power BI does support both Gen 1 and 2 DataLake, however both Power BI and Azure Analysis service does NOT support parquet file formats. You will have to convert them to a text format, like csv or other delimited format to load them from the Datalake
If you are using Databricks, and you have created a table structure based on delta tables, you can connect to those tables from Power BI using the data bricks connector. However you'll only be able to access the tables when the cluster is running.
Some outlines are on the MS Docs site > https://learn.microsoft.com/en-us/power-query/connectors/datalakestorage
You can use Azure Data Lake Storage connector in Power BI under Get Data.
For more details you can refer here
Related
I have a Google Cloud JSON key (eg. z-dexxent-3X5X02-2X5X287f780X.json) and I'd like to connect Power BI to some specific project/dataset/table in BigQuery using this JSON file.
Please, is there some step by step for this?
Thanks in advance!
I might be missing something, if your data is already in big query would use the built in Power BI Google bigquery connector
https://learn.microsoft.com/en-us/power-bi/connect-data/desktop-connect-bigquery
and if it is not yet in bigquery follow these steps
https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-json
I work for a startup where I have created several dashboards in Power BI using tables that are stored in an AWS RDS that I connect to using MySQL. To create additional columns for visualizations, I created views of the tables in MySQL and used DAX to add some extra columns for the Power BI visualizations.
However, my team now wants to use the AWS structure and build a data lake to store the raw data and a data warehouse to store the transformed data. After researching, I believe I should create the additional columns in either Athena or Redshift. My question is, which option is best for our needs?
I think the solution is to connect to the RDS using AWS Glue to perform the necessary transformations and deposit the transformed data in either Athena or Redshift. Then, we can connect to the chosen platform using Power BI. Please let me know if I am misunderstanding anything.
To give an approximate number of the number of records I'm handling, the fact tables have about 10 thousand new records every month
Thank you in advance!
Our current architecture for reporting and dashboarding is similar to the following:
[Sql Azure] <-> [Azure Analysis Services (AAS)] <-> [Power BI]
We have almost 30 Power BI Pro Licenses (no Premium Tier)
As we migrate our on-premise data feeds to ADLS Gen2 with Data Factory and Databricks (in the long run, we will dismiss SQL Azure DBs), we investigate how to connect Power BI to the delta tables.
Several approaches suggest using SQL Databricks endpoints for this purpose:
https://www.youtube.com/watch?v=lkI4QZ6FKbI&t=604s
IMHO this is nice as long as you have a few reports. What if you have, say, 20-30? Is there a middle layer between ADLS Gen2 delta tables and Power BI for a scalable and efficient tabular model? How to define measures, calculated tables, manage relationships efficiently without the hassle of doing this from scratch in every single .pbix?
[ADLS Gen2] <-> [?] <-> [Power BI]
As far as I can tell, no AAS Direct Query is allowed in this scenario:
https://learn.microsoft.com/en-us/azure/analysis-services/analysis-services-datasource
Is there a workaround to avoid the use of Azure Synapse Analytics? We are not using it, and I am afraid we will not include it in the roadmap.
Thanks in advance for your invaluable piece of advice
Is there a middle layer between ADLS Gen2 delta tables and Power BI for a scalable and efficient tabular model?
If you want to build Power BI Import Models from Delta tables without routing through Databricks SQL or Spark, you can look into the new Delta Sharing Connector for Power BI. Or run a Spark job to export the model data to a Data Lake format that Power BI/AAS can read directly.
If you want DirectQuery models, Synapse SQL Pool or Synapse Serverless would be the path, as these expose the data as SQL Server endpoints, for which Power BI and AAS support DirectQuery.
How to define measures, calculated tables, manage relationships efficiently without the hassle of doing this from scratch in every single .pbix?
Define them in an AAS Tabular Model or a Power BI Shared Data Set.
Snowflake
PowerBI
Is it possible to use different warehouse in snowflake for data refresh in Power BI than warehouse used for dashboard development?
I'd like to have separate warehouse for data refresh in order to be able to track snowflake expenses.
Let's say I'm using SF_TUTS_WH as a warehouse for development, but I'd like to use PBI_WH as warehouse for data refresh in PowerBI. Is it possible to achieve this or do I have to change snowflake warehouse in PowerBI file before I'll load it to PBI online?
FYI: https://www.snowflake.com/blog/using-sso-between-power-bi-and-snowflake/
One option which will work for you is to have the ware house name paramterized for the reports. The value of the parameter can be updated from the service based on the requirements.Please check out the below video link to see how the parameters can be updated in the power bi service.
https://youtu.be/MJVSu47iMmo
Hope this helps
Thanks ,
Prasanth
I am connecting the Power BI to Azure Data Lake store with multiple files representing multiple tables.
1)
update It is currently loading the data into the Power BI file.update
But, can I have live connection from Power BI Desktop with the Azure Data Lake Store?
2)
Can I load multiple files to represent Dimensions and Fact tables.
From desktop you can access the Azure Data Lake Store data source - just make sure you're using a recent version of Power BI Desktop Data Lake Store - Power BI.
You can join multiple queries together in Power BI desktop
The documentation for PowerBI does not list Azure Data Lake Store as a source that can be connected live as of May 2018: https://learn.microsoft.com/en-us/power-bi/refresh-data#live-connections-and-directquery-to-on-premises-data-sources.
Alternatively, you could try using Azure Stream Analytics to create a job that can copy data, and connect to the live stream, but that process might need to be manually triggered and requires data movement, which might not be ideal for your scenario. https://learn.microsoft.com/en-us/azure/data-lake-store/data-lake-store-stream-analytics
Hope this helps.
Yes, you can get data from Azure Data Lake Store from Power BI desktop application:
Also, you can join multiple queries from different tables into PBI desktop app.