SUMX over table variable breaks when adding relationships to underlying table - powerbi

Environment
I have created the two queries in Power Query
"Demo"
let
Source = Table.FromList(List.Random(1000, 20200427),Splitter.SplitByNothing()),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
#"Split Column by Position" = Table.SplitColumn(#"Changed Type", "Column1", Splitter.SplitTextByPositions({0, 5, 7}, false), {"Month", "Amount"}),
#"Changed Type2" = Table.TransformColumnTypes(#"Split Column by Position",{{"Month", type number}, {"Amount", type number}}),
ConvetToMonthNumber = Table.TransformColumns(#"Changed Type2",{{"Month", each Number.RoundDown( 12 * _) + 1, Int64.Type}})
in
ConvetToMonthNumber
and "MockCal"
let
Source = Table.FromList( List.Repeat({1..4},3), Splitter.SplitByNothing(),{"CalQuart"}),
#"Added Index" = Table.AddIndexColumn(Source, "Month", 1, 1)
in
#"Added Index"
and then in the model I have two bits of DAX
Running Amount =
CALCULATE(
SUM('Demo'[Amount]),
FILTER(ALL('Demo'[Month]), 'Demo'[Month] <= MAX('Demo'[Month]))
)
and
WEIRD Run total =
VAR CalcTable = SUMMARIZE(Demo,Demo[Month],"MonthlyRollingAmounts",[Running Amount])
VAR TotalAmount = SUMX(CalcTable,[MonthlyRollingAmounts])
RETURN IF(ISFILTERED(Demo[Month]), [Running Amount], TotalAmount)
With this I can produce the following visual:
This achieves the desired result: Create a measure that sums up its slices by month.
However, when I create a relationship off this table, the SUMX stops working as expected.
The question is: Why does adding this relationship change the behavior of the SUMX? It doesn't seem to me like it should matter at all. Help is appreciated, this one is really bending my brain.

The reason for this is that in your Running Amount measure you remove any filtering on 'Demo'[Month] but this does not propagate upstream to MockCal (which I'm assuming you're using as the first column in your visual).
In general, you want to do your date filtering on your calendar table instead of your fact table. Try this instead:
Running Amount =
CALCULATE(
SUM('Demo'[Amount]),
FILTER(ALL(MockCal), 'MockCal'[Month] <= MAX('MockCal'[Month]))
)
Your WEIRD Run total should be updated as well to use the calendar table.

Related

Filter only to show durtion

Time
Value
10/3/2022 18:21:40
correct
10/3/2022 18:22:50
incorrect
10/3/2022 18:28:00
correct
10/3/2022 18:34:00
incorrect
From the above table, I want only filter out and show on the table if the time difference between "correct" and "incorrect" is > 5 minutes
This assumes there is always only two alternating rows of correct and incorrect, and returns a column showing the duration in minutes between them. You can then filter that, since you were vague on how and what to filter
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Time", type datetime}, {"Value", type text}}),
#"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 0, 1, Int64.Type),
#"Added Custom1" = Table.AddColumn(#"Added Index", "MinuteDuration", each if [Value] = "incorrect" then null else Duration.TotalMinutes(#"Added Index"{[Index]+1}[Time]-[Time]),type number),
#"Filled Down" = Table.FillDown(#"Added Custom1",{"MinuteDuration"})
in #"Filled Down"
Here's one way to solve your problem.
Make sure your date data is sorted chronologically ascending. (this can be done in Power Query.
Create an Index Column in Power Query
Close Power Query and add the following calculated column to your table
IfNextRowOverFiveMins =
var currentRow = 'Table (2)'[index] --this should be the index on your rable
var nextTime =LOOKUPVALUE('Table (2)'[time],'Table (2)'[index],currentRow+1) --this function looks up the time for the next entry
var timeDiff = DATEDIFF('Table (2)'[time],nextTime,MINUTE) --the datediff function finds the difference in days/hours/minutes between two date values
return IF(timeDiff>=5,TRUE(),FALSE())

Query to return multiple rows based on the value in a column

I have a source table that has projects with a start date and duration in months. I'm looking to write a PowerQuery for PowerBI that will create a row for each month of the project, counting up the months. For example:
Source:
Project(string) | Date (ms timestamp) | Duration (integer)
A | Jan-2022 | 3
B | Sep-2022 | 2
Result:
Project | Date
A | Jan-2022
A | Feb-2022
A | Mar-2022
B | Sep-2022
B | Oct-2022
Not sure where to start or what this query should look like. Any ideas?
Edit: Changed sample tables to make them readable
Edit: Dates in the source table are provided in millisecond timestamp format (eg 1641024000000). My intent in the result table is to have them in a human-readable date format.
Here is one way to do this in Power Query.
Paste the code into a blank query.
Then Change the Source line so as to load your actual data table.
I used an Excel table for the source, but you may use what ever.
I also have the unix time stamp in the Source table, converting it to a PQ date in the M Code.
If all of your time stamps do not equate to the start of the month, some additional logic may be required.
Read the code comments and explore the Applied Steps to understand the algorithm
let
//Read in the Source data
Source = Excel.CurrentWorkbook(){[Name="Table27"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Project", type text}, {" Date", Int64.Type}, {" Duration", Int64.Type}}),
//convert date from unixTime in milliseconds to a PQ date
unixTime = Table.TransformColumns(#"Changed Type",{" Date", each #duration(0,0,0,_/1000)+#date(1970,1,1)}),
//add custom column with a List of the desired dates
#"Added Custom" = Table.AddColumn(unixTime, "Months", each
List.Accumulate(
{0..[#" Duration"]-1},
{},
(state,current)=> state & {Date.AddMonths([#" Date"],current)})),
//Remove unneeded columns
//Expand the list and set the data thype
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{" Date", " Duration"}),
#"Expanded Months" = Table.ExpandListColumn(#"Removed Columns", "Months"),
#"Changed Type1" = Table.TransformColumnTypes(#"Expanded Months",{{"Months", type date}})
in
#"Changed Type1"
For some reason sqlfiddle was down for me so I made an example in db-fiddle using postgres instead of ms-sql.
What you're looking to accomplish can be done with a recursive CTE, the syntax in MS-SQL is slightly different but this should get you most of the way there.
WITH RECURSIVE project_dates AS(
SELECT
start_date as starting_date,
CAST(start_date + duration*INTERVAL '1 month' as date) as end_date,
project
FROM projects
UNION
SELECT
CAST(starting_date + INTERVAL '1 month' as date),
pd.end_date,
p.project
FROM projects p
JOIN project_dates pd ON pd.project = p.project
WHERE CAST(starting_date + INTERVAL '1 month' as date) < pd.end_date
)
SELECT starting_date, project FROM project_dates
ORDER BY project, starting_date
My results using your date look as such.
You can check out my answer on db-fiddle with this link: https://www.db-fiddle.com/f/iS7uWFGwiMbEmFtNmhsiWt/0
try below
Divide your milliseconds by 86400000 and add that to 1/1/1970 to get date
Create an array based on Duration, expand to rows, add that to the start date
Remove extra columns
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
ConvertToDays = Table.TransformColumns(Source,{{"Date", each Number.RoundDown(Number.From(_) / 86400000)}}),
#"Added Custom" = Table.AddColumn(ConvertToDays, "Custom", each Date.AddDays(#date(1970,1,1),18993)),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom.1", each List.Numbers(0,[Duration])),
#"Expanded Custom.1" = Table.ExpandListColumn(#"Added Custom1", "Custom.1"),
#"Added Custom2" = Table.AddColumn(#"Expanded Custom.1", "Custom.2", each Date.AddMonths([Custom],[Custom.1]), type date),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom2",{"Date", "Duration", "Custom", "Custom.1"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Custom.2", "Date"}}),
TextDate = Table.AddColumn(#"Renamed Columns", "TextDate", each Date.ToText([Date],"MMM-yy"))
in TextDate

Working with parent child relationships in PowerBi

I am working with data that is structured in a parent-child relationship. Every level already has the rolled-up value (sum of the children). Therefore, I want Power Bi to display the value that is shown at every level (or sum on the same level) and not aggregate between parent and child. Also, the sums of the children don't always equal the parents, because some details are missing at the lower level. I do still need the parent-child relationship in PowerBi for drill through purposes. Attached is an example of how the data is structured. The table on the left is how I receive it. I have a dynamic number of levels, so ideally the solution wouldn't have a hard-coded number of levels. Any thoughts?
There's no way around having a fixed number of levels in PowerBI, but you can easily write a measure that takes the MAX rather than the SUM of the values.
You can apply these below steps to your table in Advanced Editor to get your desired output. Please replace the previous_step_name name (my frst step) accordingly to make this code functional.
let
//Your previous steps,
#"Added Index" = Table.AddIndexColumn(#"previous_step_name", "Index", 1, 1, Int64.Type),
tab_before_split = Table.ReorderColumns(#"Added Index",{"Index", "Item", "Value"}),
tab_split = Table.ExpandListColumn(Table.TransformColumns(#"tab_before_split", {{"Item", Splitter.SplitTextByDelimiter("/", QuoteStyle.Csv), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "Item"),
tab_group_by = Table.Group(#"tab_split",{"Index"}, {{"Count", each Table.RowCount(_), Int64.Type}}),
tab_new = Table.Join(tab_before_split,"Index", tab_group_by,"Index"),
#"Split Column by Delimiter" = Table.SplitColumn(tab_new, "Item", Splitter.SplitTextByDelimiter("/", QuoteStyle.Csv), {"Item.1", "Item.2", "Item.3"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"Item.1", type text}, {"Item.2", type text}, {"Item.3", type text}}),
#"Added Custom" = Table.AddColumn(#"Changed Type1", "Level", each [Count] - 1),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Count"}),
#"Changed Type2" = Table.TransformColumnTypes(#"Removed Columns",{{"Level", Int64.Type}})
in
#"Changed Type2"
Sample input-
Sample output-

how to create a DAX measure / power query process / which returns a value from another row, which has the same date & time

Please find below links to the Fact table and the overview of all tables. I would like to create a DAX measure, or new column in the Fact table ("transactions"), where:
The currency is NOT equal to EUR (e.g. "BTC"),
Giving the value equal to a opposite value of "amount", as given in the row with EUR as currency (e.g. +5. Positive for negative and vice versa),
Where the date and time of the two rows (EUR and non-EUR) have the same values (e.g. 03/11/2021 and 12:28:06)
The "type" = "trade",
In all other cases, I think it would be best to give a value of 0.
In my Fact table screenshot, I manually added the EUR_amt column in Excel to show what I would like to create
I think it's also possible to add the column, then group by time and date, such that the rows with EUR as currency with EUR_amt being 0, would be removed. All using power query. That would be even better.
(The "Currencies" table just uses the distinct values of the "currency" column in the "transactions" table, via PowerQuery. Not relevant for this question I think)
Many thanks in advance!
-YK
Fact table "transactions"
Overview of tables
Calculated Column:
EUR_amt =
IF (
OR ( transactions[type] <> "trade", transactions[currency] = "EUR" ),
0,
- LOOKUPVALUE (
transactions[amount],
transactions[Date], transactions[Date],
transactions[Time], transactions[Time],
transactions[currency], "EUR",
0
)
)
Here's one way to do this using just Power Query and the Advanced Editor
Group by data and time
Generate a custom column for each subtable based on your rules
Expand the subtables, remove those with "0", and re-order the columns
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"type", type text}, {"currency", type text}, {"Date", type date}, {"Time", type time}, {"amount", type number}}),
//Group by Date and Time
#"Grouped Rows" = Table.Group(#"Changed Type", {"Date", "Time"}, {
//Add the Eur amt column to the grouped table
{"EUR_amt", (t)=>let
//determine relevant euro amt for each sub table
eur= t[amount]{List.PositionOf(t[currency],"EUR")},
//add the column to each subtable basaed on conditions
addCol = Table.AddColumn(t, "EUR_amt",
each if [type]= "trade" and [currency]<>"EUR" then -eur else 0)
in
addCol}}),
//Expand the new table
//Filter out the 0's
//reorder the columns
#"Expanded EUR_amt" = Table.ExpandTableColumn(#"Grouped Rows", "EUR_amt", {"type", "currency", "amount", "EUR_amt"}, {"type", "currency", "amount", "EUR_amt"}),
#"Filtered Rows" = Table.SelectRows(#"Expanded EUR_amt", each ([EUR_amt] <> 0)),
#"Reordered Columns" = Table.ReorderColumns(#"Filtered Rows",{"type", "currency", "Date", "Time", "amount", "EUR_amt"})
in
#"Reordered Columns"

Power BI If Blank Then Put Value

This snippet displays the chart we are looking at and it has blanks. Instead of blanks for those dates and ID's we want to show what those values are; however, the values(Rates) may not be the same for every ID or Date. (This may happen on rare occasions.)
This snippet displays the data being pulled into Power BI from excel (Just sample data). Notice that all the dates are in order, but notice that some of the ID's do not have entries for certain dates.
I want to be able to say IF(ID is blank per this date, then put Value(Rate) that is attached to this ID).
This may not be possible to do in Power BI. We have an excel spreadsheet with similar data doing what we want but we wanted to automate by using Power BI.
Any Thoughts?
Update
I have got the measures to work in my favor, however; I need to get the total of the measure: Value Or Rate measure. I do have Value Or Rate - visual totals if I can do the same thing as the Value Or Rate measure with a total that would be great.
Please see the updated screen screenshots below.
Visual with fields
Relationships
As a measure, you could do something like below. I'm assuming that you have a model as follows:
Fact: (Date, ID, Value)
DimDate: (Date, ...)
Rates: (ID, Rate)
Dim: (ID, ...)
With these tables you'd have relationships as below:
Dim -1:N-> Fact
Dim <-1:1-> Rates
DimDate -1:N-> Fact
With the model above, you could then build a visual of:
Rows: DimDate[Date]
Columns: Dim[ID]
Values: [Value Or Rate Measure]
Value = SUM ( 'Fact'[Value] )
Rate = SUM ( 'Rates'[Rate] )
Value Or Rate Measure =
VAR Value = [Value]
RETURN
IF ( ISBLANK ( Value ), [Rate], Value )
This might not do what you want for totals - you didn't specify. So if you need visual totals you might try the following:
Value or Rate - visual totals =
SUMX (
CROSSJOIN ( VALUES ( 'Dim'[ID] ), VALUES ( 'DimDate'[Date] ) ),
[Value Or Rate Measure]
)
You could also handle this in Power Query M, assuming you have the same tables I've described above. I'm assuming each table has an associated query of the same name.
let
Source = Table.AddColumn(Dim, "Date", each DimDate[Date]),
#"Expanded Date" = Table.ExpandListColumn(Source, "Date"),
#"Merged Queries" = Table.NestedJoin(#"Expanded Date", {"id", "Date"}, Fact, {"ID", "Date"}, "Fact", JoinKind.LeftOuter),
#"Expanded Fact" = Table.ExpandTableColumn(#"Merged Queries", "Fact", {"Value"}, {"Value"}),
#"Merged Queries1" = Table.NestedJoin(#"Expanded Fact", {"id"}, Rates, {"ID"}, "Rates", JoinKind.LeftOuter),
#"Expanded Rates" = Table.ExpandTableColumn(#"Merged Queries1", "Rates", {"Rate"}, {"Rate"}),
#"Added Custom" = Table.AddColumn(#"Expanded Rates", "Value Or Rate", each if [Value] = null then [Rate] else [Value]),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Value", "Rate"})
in
#"Removed Columns"
Here, we're doing something similar, but at a table level, instead of as a measure. We crossjoin Dim[ID] and DimDate[Date] to get a dense table of all date and ID combinations. Then we left join the original Fact table and the Rates table. Then we add a column that takes [Value] if it exists, or [Rate] if [Value] is null.