How to distribute remain_qty between dist_start & dist_finish on Line & Stacked Column Chart Visual in Power BI as per excel attached. One limitation here is the remain_qty not to be distributed on weekend days [Saturdays & Sundays] & on Public Holidays. I have updated Public Holiday on Sheet 2 of the excel for quick reference. remain_qty to be distributed in between working days of dist_start & dist_finish only.
excel
Excel can be used as a source file to develop a solution in Power BI.
The visual above is currently showing remain_qty on dist_finish date and not on the whole duration between dist_start & dist_finish.
If anything you need to start is missing, please comment and I am happy to provide it.
In Power Query you can use the List.Dates function to generate a list of dates for each task, and then use Table.ExpandListColumn to create one row per task per date, and then allocate the work to each day. Here's a complete query
let
Source = Excel.Workbook(File.Contents("C:\Users\david\Downloads\excel 2.xlsx"), null, true),
Sheet1_Sheet = Source{[Item="Sheet1",Kind="Sheet"]}[Data],
#"Promoted Headers" = Table.PromoteHeaders(Sheet1_Sheet, [PromoteAllScalars=true]),
#"Removed Other Columns" = Table.SelectColumns(#"Promoted Headers",{"taskrsrc_id", "remain_qty", "dist_start", "dist_finish"}),
#"Added Custom" = Table.AddColumn(#"Removed Other Columns", "StartDate", each Date.FromText([dist_start], [Format="dd/MM/yyyy"])),
#"Duplicated Column" = Table.DuplicateColumn(#"Added Custom", "StartDate", "StartDate - Copy"),
#"Removed Columns" = Table.RemoveColumns(#"Duplicated Column",{"StartDate - Copy", "StartDate"}),
#"Added Custom1" = Table.AddColumn(#"Removed Columns", "Custom", each Date.FromText([dist_start],[Culture="en-GB"])),
#"Renamed Columns" = Table.RenameColumns(#"Added Custom1",{{"Custom", "StartDate"}}),
#"Added Custom2" = Table.AddColumn(#"Renamed Columns", "EndDate", each Date.FromText([dist_finish],[Culture="en-GB"])),
#"Removed Columns1" = Table.RemoveColumns(#"Added Custom2",{"dist_start", "dist_finish"}),
#"Added Custom4" = Table.AddColumn(#"Removed Columns1", "Duration", each Duration.Days([EndDate]-[StartDate])+1),
#"Added Custom3" = Table.AddColumn(#"Added Custom4", "Day", each List.Dates([StartDate],[Duration],#duration(1,0,0,0))),
#"Removed Columns2" = Table.RemoveColumns(#"Added Custom3",{"StartDate", "EndDate"}),
#"Expanded Days" = Table.ExpandListColumn(#"Removed Columns2", "Day"),
#"Added Custom5" = Table.AddColumn(#"Expanded Days", "Custom", each Value.Divide([remain_qty],[Duration])),
#"Renamed Columns1" = Table.RenameColumns(#"Added Custom5",{{"Custom", "Quantity"}}),
#"Removed Columns3" = Table.RemoveColumns(#"Renamed Columns1",{"remain_qty", "Duration"}),
#"Changed Type" = Table.TransformColumnTypes(#"Removed Columns3",{{"Day", type date}})
in
#"Changed Type"
Then you load it into a model like this
and it's simple to roll the work up to the month grain
Related
I need some help:
I want to scale the first table using the factors of the second table to get the third table:
The indexes in the first table are repeated (you can think of them as categories).
How can I get the third table using only the Power Query Editor of POWER BI Desktop?
This is the pbi
Thank you!
(Your sample data is wrong, with the index for the Values to Scale not matching that of Scaled Table, but anyway)
Powerquery method using unpivot, merge and then pivot. This is the code for the Values to Scale data, to convert it to be Scaled Table using data from FACTORS query
let Source = Excel.CurrentWorkbook(){[Name="ValuestoScaleData"]}[Content],
// assumes presence of query FACTORS that contains data as shown
FactorTable1 = Table.UnpivotOtherColumns(FACTORS, {"INDEX"}, "Attribute", "Value"),
FactorTable2 = Table.ReplaceValue(FactorTable1,"FACTOR ","",Replacer.ReplaceText,{"Attribute"}),
#"Added Index" = Table.AddIndexColumn(Source, "Index.1", 0, 1, Int64.Type),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Added Index", {"INDEX", "Index.1"}, "Attribute", "Value"),
#"Replaced Value" = Table.ReplaceValue(#"Unpivoted Other Columns","VALUE ","",Replacer.ReplaceText,{"Attribute"}),
#"Merged Queries" = Table.NestedJoin(#"Replaced Value", {"INDEX", "Attribute"}, FactorTable2, {"INDEX", "Attribute"}, "Table1", JoinKind.LeftOuter),
#"Expanded Table2" = Table.ExpandTableColumn(#"Merged Queries", "Table1", {"Value"}, {"Value.1"}),
#"Added Custom" = Table.AddColumn(#"Expanded Table2", "MULT", each [Value]*[Value.1]),
Rename = Table.TransformColumns(#"Added Custom",{{"Attribute", each "Scaled "&_, type text}}),
#"Removed Columns" = Table.RemoveColumns(Rename,{"Value", "Value.1"}),
#"Pivoted Column" = Table.Pivot(#"Removed Columns", List.Distinct(#"Removed Columns"[Attribute]), "Attribute", "MULT", List.Sum),
#"Sorted Rows" = Table.Sort(#"Pivoted Column",{{"Index.1", Order.Ascending}}),
#"Removed Columns1" = Table.RemoveColumns(#"Sorted Rows",{"Index.1"})
in #"Removed Columns1"
I need to convert the data as per below requirement in Power BI:
Getting the source file like below:
Need to convert each row into multiple rows based on the number of projection months (column 4) for each month. Expense month below is 1st of every month. Then divide the expense by projection month for each month planned expense.
How do I achieve this in Power BI?
You can do this transformation in the power query editor. Here below is the transformation code-
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUXJ0cgaSRgZGhroGlrpGBkCOGQgbKMXqRCsZAZkurm5wFWa6BiBNpkBsDFFhDGRGREbBVVhAVFiAVBgpxcYCAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [ID = _t, Title = _t, #"Project Start Date" = _t, #"Project Months" = _t, Expense = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}, {"Title", type text}, {"Project Start Date", type date}, {"Project Months", Int64.Type}, {"Expense", Int64.Type}}),
#"Calculated Start of Month" = Table.TransformColumns(#"Changed Type",{{"Project Start Date", Date.StartOfMonth, type date}}),
#"Added Custom" = Table.AddColumn(#"Calculated Start of Month", "Custom", each List.Range({1..[Project Months]},0)),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom.1", each [Expense]/[Project Months]),
#"Expanded Custom" = Table.ExpandListColumn(#"Added Custom1", "Custom"),
#"Added Custom2" = Table.AddColumn(#"Expanded Custom", "Custom.2", each Date.AddMonths([Project Start Date],[Custom]-1)),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom2",{"Project Start Date", "Project Months", "Expense", "Custom"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Custom.2", "Expense Month"}, {"Custom.1", "Planned Expense"}})
in
#"Renamed Columns"
Here is the final output-
I have Table A with data of specific people doing their tasks like this:
I have Table B with data of needs for specific people for different periods of time like this:
I also have additional table C with period definitions:
Period no | Date from | Date to
--------------------------------------
1 | 27/01/2021 | 24/02/2021
2 | 25/02/2021 | 24/03/2021
...
There are 2 problems here:
Someone in Table A can have Start and End dates spanning multiple periods, like for example Human B
The Start and End dates may not encompass whole Periods, they can be for example just for a couple of days. And so there's an algorithm that calculates whether this counts as a period or not:
if this is less than 5 days, than it doesn't count
if this is between 6 and 14 days, than it's 0.5 period
if it's more than 14 days, than it's 1 period
So now I want to merge Table A with Table B, to compare needs with what was delivered, for every period. The question is how to go with this?
My first thought was to add columns to Table A for Period and Quantity, to be able to group & merge over it - but what about when this deployment can span over multiple periods? Also how to implement this conditional logic for periods?
I think this works
Pull in Period definitions as Table1
Add a custom column using formula
= {Number.From([Date from])..Number.From([Date to])}
And then expand that to rows. That gives you a match for every date to every period
File .. Close and Load ... Connection
Full sample code for that part is:
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Period no", Int64.Type}, {"Date from", type date}, {"Date to", type date}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each {Number.From([Date from])..Number.From([Date to])}),
#"Expanded Custom" = Table.ExpandListColumn(#"Added Custom", "Custom"),
#"Removed Columns" = Table.RemoveColumns(#"Expanded Custom",{"Date from", "Date to"})
in #"Removed Columns"
Pull in your TableA, called Table2 here
Add custom column with similar formula and expand to rows
= {Number.From([Start of Deployment]) .. Number.From([End of Deployment])}
Now merge the other table into this one and pull in period
Click select the type and period columns and group them, pulling in the maximum and minimum dates from the new custom column
Add custom column for working duration with formula
= 1+[DayMax]-[DayMin]
Then add a custom column to apply your algo
= if [Duration]<6 then 0 else if [Duration] <15 then 0.5 else 1
Remove extra columns. Done. File ... Close and Load ... Connection
You can merge this back into your Table B as needed
Full code for this table
let Source = Excel.CurrentWorkbook(){[Name="Table2"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Type", type text}, {"Start of Deployment", type date}, {"End of Deployment", type date}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each {Number.From([Start of Deployment]) .. Number.From([End of Deployment])}),
#"Expanded Custom" = Table.ExpandListColumn(#"Added Custom", "Custom"),
#"Removed Columns" = Table.RemoveColumns(#"Expanded Custom",{"Start of Deployment", "End of Deployment"}),
#"Merged Queries" = Table.NestedJoin(#"Removed Columns",{"Custom"},Table1,{"Custom"},"Table1",JoinKind.LeftOuter),
#"Expanded Table1" = Table.ExpandTableColumn(#"Merged Queries", "Table1", {"Period no"}, {"Period no"}),
#"Grouped Rows" = Table.Group(#"Expanded Table1", {"Type", "Period no"}, {{"DayMin", each List.Min([Custom]), type number}, {"DayMax", each List.Max([Custom]), type number}}),
#"Added Custom1" = Table.AddColumn(#"Grouped Rows", "Duration", each 1+[DayMax]-[DayMin]),
#"Added Custom2" = Table.AddColumn(#"Added Custom1", "Algo", each if [Duration]<6 then 0 else if [Duration] <15 then 0.5 else 1),
#"Removed Columns1" = Table.RemoveColumns(#"Added Custom2",{"DayMin", "DayMax", "Duration"})
in #"Removed Columns1"
I have a performance monitoring table in which users are listed vertically and then months are listed across the header. These months are dynamically generated on a 12-month rolling window. At the beginning of each month, one month falls off the back of the query and another appears at the front. After beginning of month, I get the following error until I manually re-run the report:
The '<#MONTH>' column does not exist in the rowset.
Where '<#MONTH>' is the month that gets dropped off, e.g. if it is Sept 2019, it would be 'August 2018'.
I tried adding a window to the query that moved the start of the query ahead a day to try to eliminate the perceived race condition. This did not work.
Here is the M query I have currently:
let
Source = US_TOTALS,
#"Appended Query" = Table.SelectRows(Table.Combine({Source, CA_TOTALS}), each [DATELASTFULFILLMENT] >= #"This Year"),
#"Reordered Columns" = Table.ReorderColumns(#"Appended Query",{"SALESMAN", "RegionName", "DATELASTFULFILLMENT", "Total Sales", "Customer", "GROUP"}),
#"Inserted Start of Month" = Table.AddColumn(#"Reordered Columns", "StartOfMonth", each Date.StartOfMonth([DATELASTFULFILLMENT]), type date),
#"Reordered Columns1" = Table.ReorderColumns(#"Inserted Start of Month",{"SALESMAN", "RegionName", "DATELASTFULFILLMENT", "Total Sales", "StartOfMonth", "GROUP", "Customer"}),
#"Removed Columns" = Table.RemoveColumns(#"Reordered Columns1",{"Customer"}),
#"Date One" = Date.AddMonths(Date.StartOfMonth(Date.AddDays(DateTime.Date(DateTime.LocalNow()),1)),1),
#"Date Two" = Date.AddYears(Date.AddMonths(Date.StartOfMonth(Date.AddDays(DateTime.Date(DateTime.LocalNow()),1)),0),-1),
#"Filtered Rows" = Table.SelectRows(#"Removed Columns", each [DATELASTFULFILLMENT] < Date.AddMonths(Date.StartOfMonth(DateTime.Date(DateTime.LocalNow())),1) and [DATELASTFULFILLMENT] >= Date.AddYears(Date.AddMonths(Date.StartOfMonth(DateTime.Date(DateTime.LocalNow())),0),-1)),
//#"Filtered Rows" = Table.SelectRows(#"Removed Columns", each Date.From([DATELASTFULFILLMENT]) < #"Date One" and Date.From([DATELASTFULFILLMENT]) >= #"Date Two"),
#"Grouped Rows" = Table.Group(#"Filtered Rows", {"SALESMAN", "RegionName", "StartOfMonth", "GROUP"}, {{"Total", each List.Sum([Total Sales]), type number}}),
#"Reordered Columns2" = Table.ReorderColumns(#"Grouped Rows",{"SALESMAN", "RegionName", "GROUP", "StartOfMonth", "Total"}),
#"Inserted Month Name" = Table.AddColumn(#"Reordered Columns2", "Month Name", each Date.MonthName([StartOfMonth]), type text),
#"Inserted Year" = Table.AddColumn(#"Inserted Month Name", "Year", each Date.Year([StartOfMonth]), type number),
#"Merged Columns" = Table.CombineColumns(Table.TransformColumnTypes(#"Inserted Year", {{"Year", type text}}, "en-US"),{"Month Name", "Year"},Combiner.CombineTextByDelimiter(" ", QuoteStyle.None),"Month"),
#"Removed Columns2" = Table.RemoveColumns(#"Merged Columns",{"StartOfMonth", "GROUP"}),
#"Grouped Rows1" = Table.Group(#"Removed Columns2", {"SALESMAN", "Month", "RegionName"}, {{"Total", each List.Sum([Total]), type number}}),
#"Sorted Rows" = Table.Sort(#"Grouped Rows1",{{"SALESMAN", Order.Ascending}}),
#"Pivoted Columns" = Table.Pivot(#"Sorted Rows", List.Distinct(#"Sorted Rows"[Month]), "Month", "Total", List.Sum),
Columns = List.RemoveFirstN(Table.ColumnNames(#"Pivoted Columns"),2),
#"Replaced Value" = Table.ReplaceValue(#"Pivoted Columns",null,0,Replacer.ReplaceValue,Columns),
#"Changed Type" = Table.TransformColumnTypes(#"Replaced Value",List.Transform(Columns, each {_, Currency.Type })),
#"Merged Queries" = Table.NestedJoin(#"Changed Type",{"SALESMAN"},USER_MAPPING_COMBINED,{"USERNAME"},"USER_MAPPING_COMBINED",JoinKind.LeftOuter),
#"Expanded USER_MAPPING" = Table.ExpandTableColumn(#"Merged Queries", "USER_MAPPING_COMBINED", {"NAME"}, {"NAME"})
in
#"Expanded USER_MAPPING"
Expected Results:
The query refreshes as normal
Actual Results:
The query errors with The '<#MONTH>' column does not exist in the rowset.
Where '<#MONTH>' is the month that gets dropped off, e.g. if it is Sept 2019, it would be 'August 2018'.
screenshot for reference:
Ok, I'm going to take a second swing at this. If I'm way off base, I apologize. But let me bring out an example to simulate your error.
let
Seed = Number.Mod(Number.Round(Time.Second(DateTime.LocalNow())), 7) + 1,
Source = List.Generate(()=> Seed, each _ < Seed + 6, each _ + 1),
#"Converted to Table" = Table.FromList(Source, Splitter.SplitByNothing(), {"MonthNumbers"}, null, ExtraValues.Error),
#"Added Custom" = Table.AddColumn(#"Converted to Table", "MonthNames", each Date.MonthName(#date(2019, [MonthNumbers], 1))),
#"Pivoted Column" = Table.Pivot(#"Added Custom", List.Distinct(#"Added Custom"[MonthNames]), "MonthNames", "MonthNumbers", List.Sum)
in
#"Pivoted Column"
So, this creates a table that looks like so:
But every single second, the frame shifts by a month. So if you refresh the preview every second, you'll see the frame slide Jan-Jun, Feb-Jul, Mar-Aug... when December is the last month in the window, it skips back to the Jan-Jun. The point is, the columns are changing.
Now you try to load this model. It does not work!
From the time you create the model with one column set, to the time it takes to load the column set, those columns have been changed and so one of the columns isn't there any more. This is like when your month changes. When you go in and load manually, you'll update your model and things will work fine until the next change in columns. But when you're doing it with the scheduled load, that doesn't update the model, it just tries to load the data and runs into this column mismatch.
So, how do we fix it without losing this dynamic naming? Let's look at that pivot... what if we don't do it and leave our power query looking like this?
Now the column names won't change when we load it into the model. We create a matrix visualization like so, and do some refreshes:
No errors, nice dynamic headers.
So, that's the approach that I think you need. I hope it helps.
Edit: Per comment, this answer shows how to deal with a source that comes with changing column names over time, which is not the problem the asker has.
#RyanB is correct. The right approach here is to do your crosstab layout in reports rather than in data model. The right way, in general, to deal with things that change is to reify these as data, rather than as schema.
Original post below
You're looking for the 'Unpivot other columns' transform:
Select the columns whose names do not change.
Use 'Unpivot other columns' transform
Rename columns
Deal with months as a single month column
Make sure this comes before any steps that depend on the changing column names.
Here are two sample queries that are identical in code except for the source, which has differently named columns:
// query1
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("RU85FsQgCL2LdQoWUTiLL8VkJve/QliSTKF8/oK4VsO2tY8fpLyEe1aWKm3fVgvpiF7SBNLZZulQQLrD9LK339I6mAOYpoJBo5tmUOgUYNr7YwfTiUwShA/VyL/wnufhyMRqv1oHRswTJANNBshGAWN63edNkf41j4GJvzseMn67Xw==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [ID = _t, Somethingstatic = _t, #"May-2019" = _t, #"Jun-2019" = _t, #"Jul-2019" = _t, #"Aug-2019" = _t]),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(Source, {"ID", "Somethingstatic"}, "Attribute", "Value"),
#"Renamed Columns" = Table.RenameColumns(#"Unpivoted Other Columns",{{"Attribute", "Month"}}),
#"Changed Type" = Table.TransformColumnTypes(#"Renamed Columns",{{"Value", Int64.Type}, {"ID", Int64.Type}})
in
#"Changed Type"
// query 2
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("RU85FsQgCL2LdQoWUTiLL8VkJve/QliSTKF8/oK4VsO2tY8fpLyEe1aWKm3fVgvpiF7SBNLZZulQQLrD9LK339I6mAOYpoJBo5tmUOgUYNr7YwfTiUwShA/VyL/wnufhyMRqv1oHRswTJANNBshGAWN63edNkf41j4GJvzseMn67Xw==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [ID = _t, Somethingstatic = _t, #"Jun-2019" = _t, #"Jul-2019" = _t, #"Aug-2019" = _t, #"Sep-2019" = _t]),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(Source, {"ID", "Somethingstatic"}, "Attribute", "Value"),
#"Renamed Columns" = Table.RenameColumns(#"Unpivoted Other Columns",{{"Attribute", "Month"}}),
#"Changed Type" = Table.TransformColumnTypes(#"Renamed Columns",{{"Value", Int64.Type}, {"ID", Int64.Type}})
in
#"Changed Type"
My question is, is there a way to get the first/earliest date per grouping in a table and then filter the table to only include rows within the first x number of months of that first date, per grouping. Probably easiest to ask with example. Say I have the following table, and want to keep data for first 6 months of each Group:
The resulting table would look like:
Is there a way to accomplish this with DAX or M?
This seems to work:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Group", type text}, {"Date", type date}, {"Quantity", Int64.Type}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"Group"}, {{"AllData", each _, type table}}),
#"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each let DateThisRow = [AllData][Date] in Table.SelectRows([AllData],each [Date] <= Date.AddMonths(List.Min(DateThisRow),6))),
#"Removed Other Columns" = Table.SelectColumns(#"Added Custom",{"Custom"}),
#"Expanded Custom" = Table.ExpandTableColumn(#"Removed Other Columns", "Custom", {"Group", "Date", "Quantity"}, {"Group", "Date", "Quantity"})
in
#"Expanded Custom"
Starting from your original table setup, named as Table1 in Excel:
...it gives me this as an end result:
The number 6 near the end of the #"Added Custom" line in the M code above is the number of months.