I am practicing incremental refresh with independent and merged tables scenarios.
Say I have 2 tables - Sales (SalesID, ProductID, Date, Amount) and Rate (ProductID, Date, pc%)
End goal is that the Sales table must have 5th column = Sales.Amount * Rate.pc%
Incremental load needs configuring as there is large volume of data.
Scenario 1: Independent tables
I setup incremental refresh for both tables. In Power query added an extra column (newid) to both that is concatenation of ProductID+Date. In Power BI model connected the Rate table to Sales table on the newid column. To the Sales table I have added calculated column = Amount * RELATED(Rate,pc%)
I'm monitoring queries hitting the SQL server and notice that queries are sent for both tables to update their partitions. This is as expected.
Scenario 2: Merge tables in Power Query
Scenario2.Trial 1:
I setup incremental refresh for Sales table. In Power query I have merged the Rate table on ProductID and Date to retrieve the corresponding pc% value.
Observing the SQL queries there is a query for each partition of the Sales table, and the whole rate table is pulled at each iteration of the Sales partition.
Scenario2.Trial 2:
In addition to above, I setup incremental refresh on Rate table, and observed that the above behavior is the same (that is - for each partition of Sale table, the entire rate table is fetched). Plus queries are fired for rate table separately for its partitions.
Scenario2.Trial 3:
In addition to both the above, I disabled incremental load on the rate table, but kept the range start/end parameter filters on this table. And disabled load of the rate table. Now for each incremental load partition query of the Sales table, it also applies the same date filters to the rate table.
Are there any challenges to consider when using either of the above - that is - Scenario 1 or Scenario 3.Trial 3 when performing incremental refresh when the eventual goal is to merge the tables to retrieve column value from the merged table? The negative aspect of Scenario 2 is that - after the request to each Sales table partition, it fires a request for the corresponding rate table for that date range.
Related
I have a large table (FactSales) in Dw with following columns:
OrderDate|ProductID|OrderNumber|CustomerName|Amount
DateTable connects on date column to OrderDate
DimProduct connects on ProductID column to ProductID
Incremental refresh is configured on this table.
Now to improve this model in Power BI, I want to move the CustomerName to its own new dimension table (say DimCustomer). To achieve this, suppose I duplicate this fact table, then keep only the CustomerName column, remove duplicates, add an index column. Then merge the SalesFact table with DimCustomer.
At this point I'm unable to configure Incremental refresh for DimCustomer and FactSales table because the native query option is disabled (query doesn't fold).
So my intention to improve the model has the negative cost of not being able to incrementally refresh. How is this scenario (adding index column, and merge with another table) handled by Power BI engineers so that incremental refresh can be performed?
You just need to inject the RangeStart/RangeEnd parameters before the first non-foldable query step.
I was working on a data set and had few queries on how to model the data.
I have two FACT tables with the following fields
Budget table
Team Category
Team name
Service line
ID
ID_name
Employee Name
Rank
Location
Employee category
Week ending date
Hours
Revenue
Cost
Revenue or hours category
Actuals table**
Team Category
Team
Service line
ID
ID_name
Employee Name
Rank
Location
Employee category
Week ending date
Hours
Revenue
Cost
Many to many relationship exists between the tables. I have created a calendar table and a Service line table to help me filter across tables which are created as DIM tables.
The initial output I am looking for will look like the attached image
Output visual
Similar tables to be created based on Team category and Team name
I tried to follow two approaches – first one by maintaining them as separate tables and creating multiple measures for the each of the required fields. However, I was not able to get to the attached table visual
Secondly, I appended the two tables by adding an additional column named Budget and Actuals to the respective tables. Then, Hours, Revenue, Cost was unpivoted and made as the attribute. Measures were created for actuals, revenue = E, forecast, budget and variance. In this scenario, I could recreate the above output. But, could not create revenue per hour and cost per hour metrices
Can someone guide on how to move about this data model.
There are couple of other DIM tables that can be created, but that doesn't save space or is useful in filtering
I have a data model set up in PowerBI. I have a date table that's the fact table, and numerous data tables that are dimension tables.
Here is a picture of the data model. They are all connected Date[Date] <--> Table[Timestamp].
The table in question is the bottom table, RDS-WST00X-BZF-Insolation.POA.AM
Here is the table where the issue can be seen. It's like Measured Insolation doesn't see the Date Table, even though there is a connection set up in the Data Model
Here is the measure in which Measured Insolation is calculated from the RDS-WST00X-BZF-Insolation.POA.AM table
Finally, here are screenshots of both the Date Table and the RDS-WST00X-BZF-Insolation.POA.AM table
I am a beginer to Power BI
I have loaded an sql table to power BI dataset. The table keeps udating with new records frequently. How can I always pull the data of the latest five records into the dashboard. There as id no which keeps growing with the no of rows.
another question, how can I show the dashboard specific to the current working day only? There is date stamp in the sql data. How can i use it? With filter I am able to select a particular date but to set it to current day or last week etc.
Thanks in advance.
If your data doesn't contain any field which would tell you, what then latest rows are, the there's not much you can do here. Apparently you have some sort of timestamp there, so a query something like this could work as the datasource for your report:
SELECT TOP 5 field1, field2... FROM Table ORDER BY Timestamp DESC
To your second question, you can add a reletive date filter: https://learn.microsoft.com/en-us/power-bi/visuals/desktop-slicer-filter-date-range
I am fairly new to Power BI and I'm building an accounts receivable aging report with 30, 60, 90, 120 buckets. I've got all the data and totals displaying correctly in a table, until I try to sort them into the buckets.
When the user enters an as-of date in a slicer, the aging balance should adjust accordingly (this works fine) and the balance should display in the appropriate bucket (I can't get this to work). I have a separate date table related to the charges table by the date of the charge, and I need to calculate the difference between the date of the charge and the date the user picks in the slicer, which is connected to my date table.
I've tried the selectedvalue() and max() functions on my date table, but since the slicer is for dates up to the chosen date and not for one single date, selectedvalue() doesn't work, and when I compare using max(), the chosen date updates to the max(charge_date) for the particular row. So, if I choose 3/31 in the slicer but the charge date is 12/31, the max date becomes the same as the charge date, and everything falls into the current bucket. I tried dissociating the date table, but then selectedvalue() won't display at all and max() gives me the very latest date in the date table regardless of what I choose. Then, all the balances immediately go to the 120 bucket.
In my mind, what I want to do is set a variable equal to the date choice in the slicer. Then I can adjust everything in the report based on that variable. But Power Bi doesn't work that way. Does anyone have any suggestions? I can post screenshots if needed once I'm logged into my work PC.
Thank you in advance for your help.
create a Calendar table without any relation to your Transaction Table.
Keep the Date slicer and Choose After
From 01 to 29 days = CALCULATE([Due Amount]
,FILTER('BDW Account Receivable',DATEDIFF(min(Calender[Date]),'BDW Account Receivable'[DueDate],DAY)>-30
&& DATEDIFF(min(Calender[Date]),'BDW Account Receivable'[DueDate],DAY)<0))