Create an average table excluding some values - powerbi

I have a table with this structure:
Date
DeviceID
Value
01/01/2022
SensorA
1200
01/01/2022
SensorB
1300
01/01/2022
SensorC
900
02/01/2022
SensorA
500
02/01/2022
SensorB
50
02/01/2022
SensorC
39
I'm interesting in create a new table that have the average by day of all sensors values but excluding the values which are lower than the average of all sensors by day.
For example for 01/01/2022 the average of all sensors values is:
(1200+1300+900) / 3 = 1133.3
We then calculate the average sensor value for the day, excluding SensorC because its value is below 1133.3:
(1200+1300) / 2 = 1250
The final table have to looks like this:
Date
Value
01/01/2022
1250
02/01/2022
500
Is it possible? Hope you can help me
Thanks in advance!

This does exactly what you are asking for:
Avg Excl =
VAR _avg = AVERAGE ( 'Table'[Value] )
RETURN
CALCULATE (
AVERAGE ( 'Table'[Value] ) ,
'Table'[Value] > _avg
)
Result:
If you want to persist a DAX table to your model, you just use this code together with SUMMARIZE:
New_Table =
SUMMARIZE (
'Table' ,
'Table'[Date] ,
"Value",
VAR _avg = AVERAGE ( 'Table'[Value] )
RETURN
CALCULATE (
AVERAGE ( 'Table'[Value] ) ,
'Table'[Value] > _avg
)
)
Result:
For good measure, here is some Power Query code to achieve the same result. Usually this is best practice due to compression efficiency and such in Power BI. As a bonus, it also makes working with the pbix file smoother since you don't have to wait for the table to recalculate all the time when you add/remove/edit DAX components.
This solution is your best bet if this his how you actually want your table to look, since you can get rid of the original table in the model altogether and not have it in the model as a dependency. You can paste this into a blank query:
let
Source = Table.FromRecords({
[Date="2022-01-01", DeviceID="SensorA",Value=1200],
[Date="2022-01-01", DeviceID="SensorB",Value=1300],
[Date="2022-01-01", DeviceID="SensorC",Value=900],
[Date="2022-01-02", DeviceID="SensorA",Value=500],
[Date="2022-01-02", DeviceID="SensorB",Value=50],
[Date="2022-01-02", DeviceID="SensorC",Value=39]
}),
#"Changed Type" = Table.TransformColumnTypes(Source,{
{"Date", type date},
{"DeviceID", type text},
{"Value", Int64.Type}
}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"Date"}, {
{"Avg", each List.Average([Value]), type nullable number},
{"Data", each _, type table [Date=nullable date, DeviceID=nullable text, Value=nullable number]}
}),
#"Expanded Data" = Table.ExpandTableColumn(#"Grouped Rows", "Data", {"Value"}, {"Data.Value"}),
#"Added Custom" = Table.AddColumn(#"Expanded Data", "Custom", each if [Data.Value] > [Avg] then [Data.Value] else null),
#"Grouped Rows1" = Table.Group(#"Added Custom", {"Date"}, {{"Avg Value", each List.Average([Custom]), type nullable number}})
in
#"Grouped Rows1"

Since you are using the Device ID column in the table, it shows all of them separately. Can you remove the DeviceID column and try the value column as average?

Related

previous pay period using dax

I am very new to power bi and power query and while I have searched for a solution, I may be so new that I am unaware that a similar solution is here and could solve my issue but here goes.
I have been tasked with taking a table of data (which I don't have access to alter) and display in Power BI current pay period data along side data from the previous pay period (essentially doing an Excel vlookup for the previous period (each pay period is 14 days apart)) and then creating calculations to show the change in position vacancy from one pay period to the next. I only have Power Query and Power BI in which to make this happen. I created a key from different columns and then a linked a referenced table to pull the previous dates info but this didn't work once I started putting slicers on the matrix. Right now I'm trying to assign each pay period a identifier which grows by one with each later date (completed) and then create a column which reads the table and gives the previous pay period vacancy hours.
Empl Number
Sched Hours
Budg Hours
Vacant Hours
Pay Period
1
0.00
9.00
9.00
01/01/2022
2
22.5
22.5
0.00
01/01/2022
2
0.00
22.5
22.5
01/15/2022
3
0.00
37.5
37.5
01/01/2022
3
37.5
37.5
0.00
01/15/2022
3
37.5
37.5
0.00
01/29/2022
I am expecting to see something like the image below in Power BI:
Pay Period
Total Vacant Hours
Vacant Hour Change
01/01/2022
46.5
01/15/2022
60.0
13.5
01/29/2022
0.00
-60.0
02/12/2022
0.00
0.00
try to create a new table
Modelling --> New Table
previous pay period using dax = SUMMARIZE (
'Table',
'Table'[Pay Period],
"Sum Vacant", SUM ( 'Table'[Vacant Hours ] ),
"Balance Vacant",
SUM ( 'Table'[Vacant Hours ] )
- CALCULATE (
SUM ( 'Table'[Vacant Hours ] ),
FILTER (
ALL ( 'Table' ),
'Table'[Pay Period]
= CALCULATE (
MAX ( 'Table'[Pay Period] ),
FILTER (
ALL ( 'Table' ),
'Table'[Pay Period] < SELECTEDVALUE ( 'Table'[Pay Period] )
)
)
)
)
)
In Power Query: (Read the comments to understand the algorithm)
let
//Change next line to reflect your actual data source
Source = Excel.CurrentWorkbook(){[Name="Table8"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{
{"Empl Number", Int64.Type}, {"Sched Hours", type number}, {"Budg Hours", type number},
{"Vacant Hours", type number}, {"Pay Period", type date}}),
//Group by pay period and aggregate Sum of Vacant Hours
#"Grouped Rows" = Table.Group(#"Changed Type", {"Pay Period"}, {
{"Vacant Hours", each List.Sum([Vacant Hours]), type nullable number}}),
//Add a column which is a "shifted" vacant hours column
// This will put this and previous pay period on same row
#"Shifted Vacant Hours" = Table.FromColumns(
Table.ToColumns(#"Grouped Rows") & {{null} & List.RemoveLastN(#"Grouped Rows"[Vacant Hours],1)},
type table[Pay Period=date, Total Vacant Hours=number, Shifted Vacant=number]),
//add a column subtracting Shifted Vacant from Total Vacant
#"Added Custom" = Table.AddColumn(#"Shifted Vacant Hours", "Vacant Hour Change", each if [Shifted Vacant] = null then null else
[Total Vacant Hours]-[Shifted Vacant]),
//Remove the Shifted Vacant column
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Shifted Vacant"})
in
#"Removed Columns"
Edit for different grouping
If you want to have a table where the Vacant hour change is computed per employee (for each time period), then we would Group By: Empl Number, and write a custom aggregation to add the "Vacancy Change" column.
The code below assumes only a single entry per customer per pay period, as you show in your example. If there might be multiple entries per employee per pay period, minor code changes will be needed.
let
//Change next line to reflect your actual data source
Source = Excel.CurrentWorkbook(){[Name="Table8"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{
{"Empl Number", Int64.Type}, {"Sched Hours", type number}, {"Budg Hours", type number},
{"Vacant Hours", type number}, {"Pay Period", type date}}),
//Group by pay period and custom aggregaton to assess by Employee for each time period
//Assumes only a single entry per employee per time period
#"Grouped Rows" = Table.Group(#"Changed Type", {"Empl Number"}, {
{"Calcs", (t)=> let
subTable = Table.FromColumns(
Table.ToColumns(t) & {{null} & List.RemoveLastN(t[Vacant Hours])},
{"Empl Number", "Sched Hours", "Budg Hours", "Vacant Hours", "Pay Period", "Shifted Vacant"}),
#"Vacancy Change" = Table.AddColumn(subTable,"Vacant Hour Change", each [Vacant Hours] - [Shifted Vacant]),
#"Remove Shifted" = Table.RemoveColumns(#"Vacancy Change",{"Shifted Vacant"})
in #"Remove Shifted",
type table[Empl Number=Int64.Type, Sched Hours=number, Budg Hours=number, Vacant Hours=number,
Pay Period=date, Vacant Hour Change=number]
}}),
//expand the table; set columns in proper order; remove unneeded columns
#"Expanded Calcs" = Table.ExpandTableColumn(#"Grouped Rows", "Calcs", {"Sched Hours", "Budg Hours", "Vacant Hours", "Pay Period", "Vacant Hour Change"}),
#"Reordered Columns" = Table.ReorderColumns(#"Expanded Calcs",{"Empl Number", "Pay Period", "Sched Hours", "Budg Hours", "Vacant Hours", "Vacant Hour Change"}),
#"Removed Columns" = Table.RemoveColumns(#"Reordered Columns",{"Sched Hours", "Budg Hours"})
in
#"Removed Columns"

Repeat the last value over time

I have a table with power plant capacities in different years. There are only entries when something changes in the capacities. In the years not listed, the last value applies.
Plant
Year
Capacity
Cottam
2003
800
Cottam
2009
600
Cottam
2015
800
Drax
2000
600
Drax
2005
1200
Drax
2010
1800
Drax
2013
1200
Drax
2020
0
Ironbridge
2007
500
Ironbridge
2015
0
Now I would like to transform the initial table, so that I also have values for all years in between and can display them in a stacked column chart, for example. The result should look like shown in the table below. Marked in yellow are the numbers from the initial table.
You can do this easily in the Query Editor in M code.
To reproduce, paste the code below into a blank query:
let
//change next line to reflect your actual data source
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45Wcs4vKUnMVdJRMjIwMAZSFgYGSrE6qOKWQMoMU9zQFEm9S1FiBUS1AZJqhChIraERurAhSLEhhhmGxlhVG4FUQ8Q8i/LzkooyU9JTIcabAylTA6xyYGcCZWIB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Plant = _t, Year = _t, Capacity = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Plant", type text}, {"Year", Int64.Type}, {"Capacity", Int64.Type}}),
//generate Table of all years
#"All Years" = Table.FromColumns(
{List.Numbers(List.Min(#"Changed Type"[Year]), List.Max(#"Changed Type"[Year])- List.Min(#"Changed Type"[Year]) + 1 )}),
//Group by Plant
// Aggregate by joining with the All Years table and "Fill Down" to replace blanks with previous year.
// then expand the grouped column
#"Group by Plant" = Table.Group(#"Changed Type","Plant",{
{"Joined", each Table.FillDown(Table.Join(#"All Years","Column1",_,"Year",JoinKind.FullOuter),{"Capacity"})}
}),
#"Expanded Joined" = Table.ExpandTableColumn(#"Group by Plant", "Joined", {"Column1", "Capacity"}, {"Column1", "Capacity"}),
//Replace nulls with zero's
#"Replaced Value" = Table.ReplaceValue(#"Expanded Joined",null,0,Replacer.ReplaceValue,{"Capacity"}),
//Pivot on year
// then set the data types
#"Pivoted Column" = Table.Pivot(Table.TransformColumnTypes(#"Replaced Value", {{"Column1", type text}}, "en-US"),
List.Distinct(Table.TransformColumnTypes(#"Replaced Value", {{"Column1", type text}}, "en-US")[Column1]), "Column1", "Capacity"),
//set data type
#"Changed Type1" = Table.TransformColumnTypes(#"Pivoted Column",
List.Transform(List.Sort(List.RemoveFirstN(Table.ColumnNames(#"Pivoted Column"),1), Order.Ascending), each {_, Int64.Type}))
in
#"Changed Type1"
Edit Note:
Actually, to create the graph in Power BI, you do NOT want to pivot the data, so the shorter code:
let
//change next line to reflect your actual data source
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45Wcs4vKUnMVdJRMjIwMAZSFgYGSrE6qOKWQMoMU9zQFEm9S1FiBUS1AZJqhChIraERurAhSLEhhhmGxlhVG4FUQ8Q8i/LzkooyU9JTIcabAylTA6xyYGcCZWIB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Plant = _t, Year = _t, Capacity = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Plant", type text}, {"Year", Int64.Type}, {"Capacity", Int64.Type}}),
//generate Table of all years
#"All Years" = Table.FromColumns(
{List.Numbers(List.Min(#"Changed Type"[Year]), List.Max(#"Changed Type"[Year])- List.Min(#"Changed Type"[Year]) + 1 )}),
//Group by Plant
// Aggregate by joining with the All Years table and "Fill Down" to replace blanks with previous year.
// then expand the grouped column
#"Group by Plant" = Table.Group(#"Changed Type","Plant",{
{"Joined", each Table.FillDown(Table.Join(#"All Years","Column1",_,"Year",JoinKind.FullOuter),{"Capacity"})}
}),
#"Expanded Joined" = Table.ExpandTableColumn(#"Group by Plant", "Joined", {"Column1", "Capacity"}, {"Year", "Capacity"}),
//Replace nulls with zero's
#"Replaced Value" = Table.ReplaceValue(#"Expanded Joined",null,0,Replacer.ReplaceValue,{"Capacity"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Replaced Value",{{"Year", Int64.Type}, {"Capacity", Int64.Type}})
in
#"Changed Type1"
Then, in Power BI, you can generate this:
Note:
The code below presents the Table FillDown / Table Join sequence from the first code using variables and more comments. Should be easier to understand (might be less efficient, though)
...
{"Joined", each
let
//join the subtable with the All Years table
#"Joined Table" = Table.Join(#"All Years", "Column1", _, "Year", JoinKind.FullOuter),
//Fill down the Capacity column so as to fill with the "last year" data
// since that column will contain a null after the Table.Join for years with no data
#"Fill Down" = Table.FillDown(#"Joined Table",{"Capacity"})
in
#"Fill Down"
}
...
Here's how to solve this (more easily) in DAX:
Prerequisite is separate Calendar table with a 1:many relation on the year
Calendar =
SELECTCOLUMNS(
GENERATESERIES(
MIN(Plants[Year]),
MAX(Plants[Year])
),
"Year", [Value]
)
Next calculate the Last Given Capacity per year
Last Given Capacity =
VAR current_year =
MAX(Calendar[Year])
VAR last_capacity_year =
CALCULATE(
MAX(Plants[Year]),
'Calendar'[Year] <= current_year
)
RETURN
CALCULATE(
MAX(Plants[Capacity]),
Calendar[Year] = last_capacity_year
)
Finally put it all together in a Stacked Column Chart with
X-axis: 'Calendar'[Year]
Y-axis: [Last Given Capacity]
Legend: 'Plants'[Plant]

In Dax, how can i count values in one column that equal the value of another column?

I have two columns, i want a table that shows the number of "Assign Date" in "Week Start" so for "Week Start" of 1/1/2022 it should be 0, for "Week Start" of 1/7/2022, it should be 2, and it should be 1 for 1/14/2022 and 1/21/2022.
I have two date column
Week Start
Assign Date
1/1/2022
1/8/2022
1/8/2022
1/8/2022
1/15/2022
1/15/2022
1/22/2022
1/22/2022
I want one date column and one count column
Week Start
Assign Count
1/1/2022
0
1/8/2022
2
1/15/2022
1
1/22/2022
1
I am very new to DAX and i assume that i am over complicating the solution but i can't figure out where to start. Because i am learning DAX, i would like to get this in a DAX measure.
Or this measure:
Assign Count :=
VAR ThisWeekStart =
MIN( Table1[Week Start] )
RETURN
0
+ COUNTROWS(
FILTER(
ALL( Table1 ),
Table1[Assign Date] = ThisWeekStart
)
)
which you can place in a visual together with the Week Start field.
There may be more efficient M-Code, but what I did here was
to use List.Accumulate to count the number of entries that were in the correct range: >=Week Start and <Week Start + 7 days
M Code
let
Source = Excel.CurrentWorkbook(){[Name="Table3"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Week Start", type date}, {"Assign Date", type date}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Assign Count",
each List.Accumulate(
#"Changed Type"[Assign Date],
0,
(state, current)=>
if current >=[Week Start] and current < Date.AddDays([Week Start],7) then state +1 else state)),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Assign Date"})
in
#"Removed Columns"

Table with all dates between last sales day and defined N periods

How to get Dates for N last full months. I want the last month to be determined by Sales amount.
This is the whole table for the example.
The expected result is a calculated table of Date column from 2020-05-01 to 2020-07-31. Looks like this:
Date
2020-05-01
2020-05-02
2020-05-03
…
2020-07-29
2020-07-30
2020-07-31
What have I tried? First, a measure to get the last date with sales:
MaxDate =
CALCULATE(
EOMONTH( MAX( T[Date] ), 0),
ALL( T ),
T[Amount] > 0
)
And the calculated table:
T_Range =
var a = [MaxDate]
var b = DATESINPERIOD( T[Date] , a, -3, MONTH )
return
b
But it returns only 3 days, not the whole range from 2020-05-01 to 2020-07-31.
The table to reproduce the problem:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("ZcrLCQAgDATRXnJWSNZ/LWL/bShIQFyY02PmFCg0qp0kiMkKTmBKTJmpROCjyldj6pceGb+YkhgJXNYG", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [Date = _t, Amount = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Amount", Int64.Type}})
in
#"Changed Type"
Please check if this helps -
Created a new column with 'Max date' measure you provided.
Then crated min date column with below DAX.
Min Date = EDATE([MaxDate],-3)+1
Created new table using below.
T_Range = CALENDAR(MAX(T[Min Date]),MAX(T[Max Date]))

increase date column based on conditions

I am having some issues with how to approach my query so any help would be greatly appreciated.
I have a date column that I need to increase based on two other columns values.
e.g. Date Reported column - 17/12/2018
If my Impact Column = "Urgent" and my Department = "Stores" I would need to increase my Date Reported Column to 18/12/2018
However if my Impact Column = "Standard" and my Department = "Floor" I would need to increase my Date Reported Column to 20/12/208
I would ideally like to not touch the original Date Reported Column but move this new value to another column.
So Far I have created a custom column and this is my code however it doesnt work.
AmendedDateReported = if(And(SurveyCorrectiveAction[Impact] = "Urgent", SurveyCorrectiveAction[LookUp] = "Stores"), Date.AddDays([DateReported],1),Blank ())
Thanks
Paula
Updated code, The formula seems to be pulling ok but the date part wont update:
#"Sorted Rows" = Table.Sort(Source,{{"DateReported", Order.Ascending}}),
#"Changed Type" = Table.TransformColumnTypes(#"Sorted Rows",{{"DateReported", type date}}),
#"Sorted Rows1" = Table.Sort(#"Changed Type",{{"DateReported", Order.Descending}}),
#"Added Custom" = Table.AddColumn(#"Sorted Rows1", "Date Repaired", each ""),
#"Changed Type1" = Table.TransformColumnTypes(#"Added Custom",{{"Date Repaired", type text}}),
#"Duplicated Column" = Table.DuplicateColumn(#"Changed Type1", "DateReported", "DateReported - Copy"),
#"Renamed Columns" = Table.RenameColumns(#"Duplicated Column",{{"DateReported - Copy", "AmendedDateReported"}}),
#"Merged Amendments" = Table.NestedJoin(#"Renamed Columns",{"Impact", "Department"},TLU_FaultTimeScales,{"Impact", "Department"},"TLU_FaultTimeScales",JoinKind.LeftOuter),
#"Expanded Amendments" = Table.ExpandTableColumn(#"Merged Amendments", "TLU_FaultTimeScales", {"Amendment Day"}, {"Amendment Day"}),
AmendedDateReported = Table.AddColumn(#"Expanded Amendments", "AmendedDateReported", each try Date.AddDays([DateReported],[Amendment Day]) otherwise [DateReported], type date)
in
#"Renamed Columns"
You could try:
AmendedDateReported =
Table.AddColumn(
#"Previous Step",
"Amended Date Reported",
each Date.AddDays(
[Date Reported],
if [Impact] = "Urgent" and [Department] = "Stores" then 1
else if [Impact] = "Standard" and [Department] = "Floor" then 3
else 0
),
type date
)
If you have several combinations of Impact / Department which have variable effect on amending the date, it would make more sense to put those in a separate table:
+----------+------------+----------------+
| Impact | Department | Amendment Days |
+----------+------------+----------------+
| Urgent | Stores | 1 |
| Standard | Floor | 3 |
+----------+------------+----------------+
You can then join this table to retrieve the amendment days:
#"Merged Amendments" = Table.NestedJoin(#"Previous Step",{"Impact", "Department"},tblAmendments,{"Impact", "Department"},"tblAmendments",JoinKind.LeftOuter),
#"Expanded Amendments" = Table.ExpandTableColumn(#"Merged Amendments", "tblAmendments", {"Amendment Days"}, {"Amendment Days"}),
AmendedDateReported = Table.AddColumn(#"Expanded Amendments", "Amended Date Reported", each try Date.AddDays([Date Reported],[Amendment Days]) otherwise [Date Reported], type date)
in
AmendedDateReported
Remember to update the final variable name after the in clause.