When a query can be folded - that is - when the last step of the power query transformations shows the native SQL query, then Power BI will only pull specific columns from the SQL table.
If a query cannot be folded, and say one of the prior transformations is to remove all columns other than orderid, date, saleamt; then does Power BI fetch the entire table (all columns)?
PQ streams data. If folding cannot occur, then each row is streamed as required. You can learn everything you want to know from Ben Gribaudo: https://bengribaudo.com/blog/2020/08/26/5417/how-power-query-thinks
PQ may also reorder your steps to optimise folding.
Related
I have made table as follows (https://apexinsights.net/blog/convert-date-range-to-list):
In this scenario suppose I configure Incremental refresh on the Start Date column, then will Power BI support this correctly. I am asking because - say the refresh is for last 2 days or last 2 months, then it will fetch the source rows, and apply the transform to the partition. But my concern is that I will have to put the date param filter on Start date prior to non folding steps so that the query folds (alternatively power query will auto apply date filter so that query can fold).
So when it pulls the data based on start date and apply the transforms then I'm not able to think clearly about what kind of partitions it will create. Whether it is for Start date or for expabded date. Is query folding supported in this scenario?
This is a quite complicated scenario, where I would probably just avoid adding incremental refresh.
You would have to use the RangeStart/RangeEnd parameters twice in this query. Once that gets folded to the data source to retrieve ranges that overlap with the [RangeStart,RangeEnd) interval and a second time after expanding the ranges to filter out individual rows that fall outside [RangeStart,RangeEnd).
I need help on this issue as i don't have any experience in Power Bi. I want to join 2 table in Power Bi where it have the same column which is Part_Number. How can i make this 2 table to match by Part Number and return the value?
Recon Table
Inventory Table
I would like to have Part Number, Part Name, QTY, Total Quantity as the result. Hope that i can the clarification i need. Thanks a lot!
For this case you simply must merge the tables. It doesn't look like you have done a lot of research on the matter though, so it's hard to understand exactly what you need help with.
To merge your two tables in Power Query, I would right click in the left hand side menu and select Merge Queries as New.
After that you simply follow the on-screen instructions and select your two tables and their respective key columns. After merging you can choose to disable load of your two original tables to save space in your data model, but this depends on your requirements.
If this was my data model, I would think on why joining these tables are necessary, instead of using these two tables as fact tables, and creating a third table to handle the part number dimension with associated part metadata.
Read the docs: Merge queries in Power Query
I am trying to create a table that is a list of all possible combinations between two tables: products and companies. I'm building a dashboard in Power BI and the data comes from SQL queries. I have the following list structures for the Products table and the Companies table:
and my desired output for analysis is:
There is nothing that relates the companies to the products, and I'm trying to get a list of all of the products for each company. Can I do this in Power BI? If not, is it possible in SQL (there is nothing to join on)? Thank you for your help!
You can do this with a 'hacked' join, or a cross join. I prefer the former from a process POV, but cannot speak to speed or efficiency.
Using a Join ( Merge in PQ )
Create a new column 'DummyKey' with a value of 1 on each table.
Merge both tables using your 'DummyKey' columns.
Complete the Join process and choose the columns you want to bring through.
Cross Joins in PowerQuery according to MS
I think this 'cross join' is only technically right and does not provide future flexibility.
Both methods will get you to the same end point, and can be done in SQL or PQ.
I am attempting to build a data model in Power BI from a data mart (star schema) in SQL Server. This data mart has a fact table and several dimension tables. One of the dimension tables is a date table. I want to load all rows from the fact table. However, I only want to load a subset of the date table. In particular, I want those dates (rows) between the min and max dates in my fact table. This way, when I create slicers and such, I don't have unnecessary dates appearing.
In other BI tools (e.g., Qlik Sense), the usual solution is to first load the fact table into memory, compute and load its min/max dates into another table (also in-memory), set variables from this other table, load the date dimension table (into memory) based on the min/max variables, and finally drop the temporary table from memory so that it doesn't stay in the model and cause problems. This seems like the most efficient solution to me. It only reads the required (as opposed to unnecessary) data from the source dimension table, it doesn't need to perform any joins in the source, and it only reads each table once (as opposed to 2+ times).
How can I achieve this in Power BI? Or, more importantly, is this solution method even possible in Power BI?
I found this solution, but it seems inefficient, as it creates 2 queries (instead of just 1) for the min/max and, moreover, it performs the dimension table filtering after all rows have already been fetched from the source. (In my particular case, this isn't too bad. But, it could be problematic in other situations in which my dimension table is large.)
I'm attempting to create a shared date dimensions between two fact tables in Power BI, based off of a relational data source.
Currently, if I include an unrelated dimension in the report, I get numbers duplicated across multiple rows, where they don't really apply.
I'm wondering if there is any way to tell Power BI that certain dimensions cannot be used with certain fact tables, similar to using IgnoreUnrelatedDimensions in SSAS.
Currently the only solution I can find is to create a separate date dimension, so that the two fact tables have no relationship that could be used to join them, however this would mean forfeiting the ability to do any time based comparisons.
Create a combined view of the fact tables with only compatible columns to be used for time based comparison:
In Query Editor, create new queries for your fact tables by
referencing i.e. right click original query and select "Reference".
Then in those "copies" cut out the incompatible dimensions.
Rename columns to align terminology (e.g. Sales Date ==> Transaction Date, Payment Date ==> Transaction Date).
Use "Merge Queries" function to combine the copies using Full Outer Join.
Join this merged view to your date dimension