I have a table with received time like the below:
As you see, the frequency of received data in some rows is different and they are 1000ms, 1001ms, 998ms.
How can I calculate the average frequency of received time in ms?
I suggest using Power Query to
add a column that is the original date_time column offset by 1
Then add another column showing the difference between current row and previous row
This method, at least in M Code, is faster than using an Index column to refer to previous row
Then you can do your mathematical analyses.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("dc27DcAgDAXAVRA1An9iO2EVxP5rgEQZXn3FjZGjsTUh4cRdqctTwy3P8heD4lv8KgHl3RJX+dCjBIXRo3JkLg==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [date_time = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"date_time", type datetime}}),
//add column containing previous rows data
//usually faster than using and INDEX column
prevList = {null} & List.RemoveLastN(#"Changed Type"[date_time]),
tbl1 = Table.FromColumns(
{#"Changed Type"[date_time],prevList},
{"date_time","Prev Row"}
),
#"Added Custom" = Table.AddColumn(tbl1, "Difference", each /*if [Prev Row] = null then 0
else*/ Duration.TotalSeconds([date_time] - [Prev Row])),
#"Changed Type1" = Table.TransformColumnTypes(#"Added Custom",{
{"date_time", type datetime}, {"Prev Row", type datetime}, {"Difference", type number}})
in
#"Changed Type1"
In PowerQuery add an Index column to your data so that you can address singe rows. Then with DAX add a calulated column to the table:
Milliseconds =
VAR NextIndex = 'Table'[Index] + 1
VAR MaxIndex = MAX(Table[Index])
VAR Difference =
IF(
'Table'[Index] = MaxIndex,
BLANK(),
'Table'[date_time] -
CALCULATE(
VALUES('Table'[date_time]),
Filter(
ALL('Table'),
'Table'[Index] = NextIndex
)
)
)
RETURN
CONVERT(ABS(Difference * 3600 * 24 * 1000), INTEGER)
Now you are looking for AVERAGE( Milliseconds ).
Note: Things would have been easier if you provided copyable data instead of a screenshot.
Related
I have a table with power plant capacities in different years. There are only entries when something changes in the capacities. In the years not listed, the last value applies.
Plant
Year
Capacity
Cottam
2003
800
Cottam
2009
600
Cottam
2015
800
Drax
2000
600
Drax
2005
1200
Drax
2010
1800
Drax
2013
1200
Drax
2020
0
Ironbridge
2007
500
Ironbridge
2015
0
Now I would like to transform the initial table, so that I also have values for all years in between and can display them in a stacked column chart, for example. The result should look like shown in the table below. Marked in yellow are the numbers from the initial table.
You can do this easily in the Query Editor in M code.
To reproduce, paste the code below into a blank query:
let
//change next line to reflect your actual data source
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45Wcs4vKUnMVdJRMjIwMAZSFgYGSrE6qOKWQMoMU9zQFEm9S1FiBUS1AZJqhChIraERurAhSLEhhhmGxlhVG4FUQ8Q8i/LzkooyU9JTIcabAylTA6xyYGcCZWIB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Plant = _t, Year = _t, Capacity = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Plant", type text}, {"Year", Int64.Type}, {"Capacity", Int64.Type}}),
//generate Table of all years
#"All Years" = Table.FromColumns(
{List.Numbers(List.Min(#"Changed Type"[Year]), List.Max(#"Changed Type"[Year])- List.Min(#"Changed Type"[Year]) + 1 )}),
//Group by Plant
// Aggregate by joining with the All Years table and "Fill Down" to replace blanks with previous year.
// then expand the grouped column
#"Group by Plant" = Table.Group(#"Changed Type","Plant",{
{"Joined", each Table.FillDown(Table.Join(#"All Years","Column1",_,"Year",JoinKind.FullOuter),{"Capacity"})}
}),
#"Expanded Joined" = Table.ExpandTableColumn(#"Group by Plant", "Joined", {"Column1", "Capacity"}, {"Column1", "Capacity"}),
//Replace nulls with zero's
#"Replaced Value" = Table.ReplaceValue(#"Expanded Joined",null,0,Replacer.ReplaceValue,{"Capacity"}),
//Pivot on year
// then set the data types
#"Pivoted Column" = Table.Pivot(Table.TransformColumnTypes(#"Replaced Value", {{"Column1", type text}}, "en-US"),
List.Distinct(Table.TransformColumnTypes(#"Replaced Value", {{"Column1", type text}}, "en-US")[Column1]), "Column1", "Capacity"),
//set data type
#"Changed Type1" = Table.TransformColumnTypes(#"Pivoted Column",
List.Transform(List.Sort(List.RemoveFirstN(Table.ColumnNames(#"Pivoted Column"),1), Order.Ascending), each {_, Int64.Type}))
in
#"Changed Type1"
Edit Note:
Actually, to create the graph in Power BI, you do NOT want to pivot the data, so the shorter code:
let
//change next line to reflect your actual data source
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45Wcs4vKUnMVdJRMjIwMAZSFgYGSrE6qOKWQMoMU9zQFEm9S1FiBUS1AZJqhChIraERurAhSLEhhhmGxlhVG4FUQ8Q8i/LzkooyU9JTIcabAylTA6xyYGcCZWIB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Plant = _t, Year = _t, Capacity = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Plant", type text}, {"Year", Int64.Type}, {"Capacity", Int64.Type}}),
//generate Table of all years
#"All Years" = Table.FromColumns(
{List.Numbers(List.Min(#"Changed Type"[Year]), List.Max(#"Changed Type"[Year])- List.Min(#"Changed Type"[Year]) + 1 )}),
//Group by Plant
// Aggregate by joining with the All Years table and "Fill Down" to replace blanks with previous year.
// then expand the grouped column
#"Group by Plant" = Table.Group(#"Changed Type","Plant",{
{"Joined", each Table.FillDown(Table.Join(#"All Years","Column1",_,"Year",JoinKind.FullOuter),{"Capacity"})}
}),
#"Expanded Joined" = Table.ExpandTableColumn(#"Group by Plant", "Joined", {"Column1", "Capacity"}, {"Year", "Capacity"}),
//Replace nulls with zero's
#"Replaced Value" = Table.ReplaceValue(#"Expanded Joined",null,0,Replacer.ReplaceValue,{"Capacity"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Replaced Value",{{"Year", Int64.Type}, {"Capacity", Int64.Type}})
in
#"Changed Type1"
Then, in Power BI, you can generate this:
Note:
The code below presents the Table FillDown / Table Join sequence from the first code using variables and more comments. Should be easier to understand (might be less efficient, though)
...
{"Joined", each
let
//join the subtable with the All Years table
#"Joined Table" = Table.Join(#"All Years", "Column1", _, "Year", JoinKind.FullOuter),
//Fill down the Capacity column so as to fill with the "last year" data
// since that column will contain a null after the Table.Join for years with no data
#"Fill Down" = Table.FillDown(#"Joined Table",{"Capacity"})
in
#"Fill Down"
}
...
Here's how to solve this (more easily) in DAX:
Prerequisite is separate Calendar table with a 1:many relation on the year
Calendar =
SELECTCOLUMNS(
GENERATESERIES(
MIN(Plants[Year]),
MAX(Plants[Year])
),
"Year", [Value]
)
Next calculate the Last Given Capacity per year
Last Given Capacity =
VAR current_year =
MAX(Calendar[Year])
VAR last_capacity_year =
CALCULATE(
MAX(Plants[Year]),
'Calendar'[Year] <= current_year
)
RETURN
CALCULATE(
MAX(Plants[Capacity]),
Calendar[Year] = last_capacity_year
)
Finally put it all together in a Stacked Column Chart with
X-axis: 'Calendar'[Year]
Y-axis: [Last Given Capacity]
Legend: 'Plants'[Plant]
I have a lot to learn of Power BI and I am currently stuck with a column I need to add in my Calendar table.
I have to calculate a workday column that should flag each day of the month with it's workday number (from 1 to the maximum number of working days for that month).
I was able to find a solution online with RANKX (I have a column, NonWorkday that flags with 1 for all days that are either weekends or holidays) but it's not complete:
Workday =
VAR CurrentMonth = Calendar[MonthYear]
VAR MonthTable = FILTER( ALL( Calendar ), Calendar[NonWorkDay] = 0 && Calendar[MonthYear] = CurrentMonth )
RETURN
IF( Calendar[NonWorkDay] = 1, BLANK(), RANKX(MonthTable, CALCULATE(AVERAGE(Calendar[DayofMonth])), ,ASC))
What I'm missing is that, for weekends and holidays I don't want blank, I want it to have the same workday number as the previous workday and if the month starts with a non-working day, that will be 0. An example image below:
It would be great if anyone can help me with a solution or point me in the right direction, either DAX or M query.
Thank you!
LE: Adding MVE code, as per #Marco_CH request:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("bdJBDoMwDATAv+RciXgTSPIW1P9/oxTbar34wGUQCWvveRbZZEMFyqvI9dTyfp0FGbYMe4b7D+v94otHRNw4IrYbZzxTcWUoNVWJp3ZVRN1VW9RDtUcdqnu8zfRIlZJN1Rl1qS6aly2hEuscQelEJwnamXPLmQKKTg68OR0daHeiswNlFB0eKKToREAbNG41Z05pA9zwbBYiwtqKZ7N6xG5txf8PdGtrgiN+vltb8SzQiji8rUg6IfGq6W3NtKVKsZa3FUmr5CAW7yuSVgmls57Iiv/h9ak589a6NxZZqyijM4X0snHK4Y2Nh0xvbMqc8prg+wM=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Date = _t, NonWorkDay = _t, WantedWorkday = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"NonWorkDay", Int64.Type}, {"WantedWorkday", Int64.Type}})
in
#"Changed Type"
LE2: My final calculation based on the answered provided by #msta42a:
WD =
var day ='Calendar'[Date]
VAR CurrentMonth = Calendar[MonthYear]
return
CALCULATE( COUNTROWS('Calendar'), FILTER('Calendar', Calendar[MonthYear] = CurrentMonth && 'Calendar'[Date] <= day && 'Calendar'[NonWorkDay] <> 1)) +0
Results:
You can calculate this like "running count/total" here my example (in your code you should add a clause for month/year)
Workday =
var day ='Calend'[Date]
return
CALCULATE( COUNTROWS('Calend'), FILTER('Calend', 'Calend'[Date] <= day && 'Calend'[Weekend]="wd" && Calend[Nonworking] <> 0)) +0
How to get Dates for N last full months. I want the last month to be determined by Sales amount.
This is the whole table for the example.
The expected result is a calculated table of Date column from 2020-05-01 to 2020-07-31. Looks like this:
Date
2020-05-01
2020-05-02
2020-05-03
…
2020-07-29
2020-07-30
2020-07-31
What have I tried? First, a measure to get the last date with sales:
MaxDate =
CALCULATE(
EOMONTH( MAX( T[Date] ), 0),
ALL( T ),
T[Amount] > 0
)
And the calculated table:
T_Range =
var a = [MaxDate]
var b = DATESINPERIOD( T[Date] , a, -3, MONTH )
return
b
But it returns only 3 days, not the whole range from 2020-05-01 to 2020-07-31.
The table to reproduce the problem:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("ZcrLCQAgDATRXnJWSNZ/LWL/bShIQFyY02PmFCg0qp0kiMkKTmBKTJmpROCjyldj6pceGb+YkhgJXNYG", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [Date = _t, Amount = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Amount", Int64.Type}})
in
#"Changed Type"
Please check if this helps -
Created a new column with 'Max date' measure you provided.
Then crated min date column with below DAX.
Min Date = EDATE([MaxDate],-3)+1
Created new table using below.
T_Range = CALENDAR(MAX(T[Min Date]),MAX(T[Max Date]))
I am new with power BI and need your help.
I have 3 columns which are:
date1
reviseDate
Shipped_Date
I need to compare column reviseDate to shipped_date if it is <> or =.
How to I do it, if reviseDate column is blank, must refer on column Date1 to compare with shipped_Date column?
Try this as a calculated column:
Status =
VAR DueDate =
IF(ISBLANK(Table1[Revised]), [Date1], [Revised])
RETURN
SWITCH(
TRUE(),
ISBLANK(DueDate), BLANK(),
DueDate = Table1[Shipped_Date], "On Time",
DueDate < Table1[Shipped_Date], "Late",
DueDate > Table1[Shipped_Date], "Early"
)
This defines the date you want to compare as a variable that we use to check different conditions.
The SWITCH(TRUE(),...) is a useful construction that returns the specified result for the first condition in the list that evaluates to TRUE().
You can define a new calculated column like this:
Status = IF (AND(ISBLANK(Table1[Date1]); ISBLANK(Table1[ReviseDate])); BLANK();
IF (Table1[Shipped_Date] = IF(ISBLANK(Table1[ReviseDate]); Table1[Date1]; Table1[ReviseDate]); "On Time";
IF (Table1[Shipped_Date] > IF(ISBLANK(Table1[ReviseDate]); Table1[Date1]; Table1[ReviseDate]); "Late";
IF (Table1[Shipped_Date] < IF(ISBLANK(Table1[ReviseDate]); Table1[Date1]; Table1[ReviseDate]); "Early"; BLANK())
)
)
)
The expression IF(ISBLANK(Table1[ReviseDate]); Table1[Date1]; Table1[ReviseDate]) will return Date1 value in case ReviseDate is empty. Then it is matter of comparing this reference date with Shipped_Date and returning appropriate status.
If you want to make status column using Power Query Editor, you can use Add Column -> Conditional Column. Make one helper column, named ReferenceDate or something, which will compute the date we need to use for comparison:
Then add the actual Status column like this:
The M code will look like this (the data is embedded in it):
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("bY7RCQAhDEN36bcUUxTrLOL+aygenCkI/WjymtAxxKp2tQyXJHuunGnDFmFj+HldAQqUGzC89nPlsdYZuiITrL98PRRqQUkYCyqZCw==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [Date1 = _t, ReviseDate = _t, Shipped_Date = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date1", type date}, {"ReviseDate", type date}, {"Shipped_Date", type date}}),
#"Added Conditional Column" = Table.AddColumn(#"Changed Type", "ReferenceDate", each if [ReviseDate] = null then [Date1] else [ReviseDate]),
#"Added Conditional Column1" = Table.AddColumn(#"Added Conditional Column", "Status", each if [ReferenceDate] = null then "" else if [ReferenceDate] = [Shipped_Date] then "On Time" else if [ReferenceDate] < [Shipped_Date] then "Late" else if [ReferenceDate] > [Shipped_Date] then "Early" else null),
Status = #"Added Conditional Column1"{2}[Status]
in
Status
Depending on your data source, you can even modify the query and retrieve this value from the database (e.g. with Transact-SQL query if your data source is SQL Server).
In PowerBI, I have a simple table with 3 columns:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("TYzBCcAwDAN38dugyk5bdxaT/deICzXN77hDyhSKCkHYwafwbJyaYiUc9I4XaH/1MgHeXQO+bcf7ux0XW3x5Lg==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [id = _t, start = _t, end = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"id", Int64.Type}, {"start", type date}, {"end", type date}})
in
#"Changed Type"
From which I can create the following visual
My challenge is to calculate the total duration in days of the times series based on filter selected above. Any help would be appreciated.
I have tried the following DAX formula but it gives me crazy results as shown above.
YTDDuration =
var start_Date=FIRSTDATE(ALLSELECTED('CALENDAR'[Date]))
var end_Date=LASTDATE(ALLSELECTED('CALENDAR'[Date]))
var current_Start=MAX(Table2[start])
var current_end=MAX(Table2[end])
var bigif=IF(current_end>start_Date&¤t_Start<end_Date,DATEDIFF(MAX(start_Date,current_Start),MIN(end_Date,current_end),DAY),0)
RETURN
CALCULATE(SUMX(Table2, bigif),FILTER(ALL(Table2), Table2[start] <= max(Table2[end])))
Expected output would be:
The key here is to account for gaps in dates and consolidate overlapping dates.
You can iterate through your calendar table and count the number of days where that day falls into one of the id time periods.
YTDDuration =
var current_Start = CALCULATE(MIN(Table2[start]), ALL(Table2))
var current_end = MAX(Table2[end])
RETURN
SUMX(
FILTER('CALENDAR', 'CALENDAR'[Date] <= current_end),
MAXX(ALL(Table2), IF([Date] > [start] && [Date] <= [end], 1, 0))
)
This starts at the minimal start date and adds 1 for each day where that Date is between start and end for some row in Table2. If none of the rows contains that Date, the max is over just zeros and returns zero. If one or more matches, you get a max of one and we count that day.
YTD Duration by end: