how to create a DAX measure / power query process / which returns a value from another row, which has the same date & time - powerbi

Please find below links to the Fact table and the overview of all tables. I would like to create a DAX measure, or new column in the Fact table ("transactions"), where:
The currency is NOT equal to EUR (e.g. "BTC"),
Giving the value equal to a opposite value of "amount", as given in the row with EUR as currency (e.g. +5. Positive for negative and vice versa),
Where the date and time of the two rows (EUR and non-EUR) have the same values (e.g. 03/11/2021 and 12:28:06)
The "type" = "trade",
In all other cases, I think it would be best to give a value of 0.
In my Fact table screenshot, I manually added the EUR_amt column in Excel to show what I would like to create
I think it's also possible to add the column, then group by time and date, such that the rows with EUR as currency with EUR_amt being 0, would be removed. All using power query. That would be even better.
(The "Currencies" table just uses the distinct values of the "currency" column in the "transactions" table, via PowerQuery. Not relevant for this question I think)
Many thanks in advance!
-YK
Fact table "transactions"
Overview of tables

Calculated Column:
EUR_amt =
IF (
OR ( transactions[type] <> "trade", transactions[currency] = "EUR" ),
0,
- LOOKUPVALUE (
transactions[amount],
transactions[Date], transactions[Date],
transactions[Time], transactions[Time],
transactions[currency], "EUR",
0
)
)

Here's one way to do this using just Power Query and the Advanced Editor
Group by data and time
Generate a custom column for each subtable based on your rules
Expand the subtables, remove those with "0", and re-order the columns
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"type", type text}, {"currency", type text}, {"Date", type date}, {"Time", type time}, {"amount", type number}}),
//Group by Date and Time
#"Grouped Rows" = Table.Group(#"Changed Type", {"Date", "Time"}, {
//Add the Eur amt column to the grouped table
{"EUR_amt", (t)=>let
//determine relevant euro amt for each sub table
eur= t[amount]{List.PositionOf(t[currency],"EUR")},
//add the column to each subtable basaed on conditions
addCol = Table.AddColumn(t, "EUR_amt",
each if [type]= "trade" and [currency]<>"EUR" then -eur else 0)
in
addCol}}),
//Expand the new table
//Filter out the 0's
//reorder the columns
#"Expanded EUR_amt" = Table.ExpandTableColumn(#"Grouped Rows", "EUR_amt", {"type", "currency", "amount", "EUR_amt"}, {"type", "currency", "amount", "EUR_amt"}),
#"Filtered Rows" = Table.SelectRows(#"Expanded EUR_amt", each ([EUR_amt] <> 0)),
#"Reordered Columns" = Table.ReorderColumns(#"Filtered Rows",{"type", "currency", "Date", "Time", "amount", "EUR_amt"})
in
#"Reordered Columns"

Related

Visualization in Power BI

I am new to Power BI Desktop and Power BI Query. The left part is the data, and the right part is my desired output in Power BI. I tried to use the Matrix Report Visual to format my data like the table on the right, but it only allowed me to select the Product column in the row. I hope to get the desired output without creating separate tables for each quarter.
You could bring your data into powerquery, and use this code to transform the data. You'd have to modify it to show your actual table name (Table1) and the name of the columns as appropriate. It basically groups on index/product to find the most recent target sale, expands the data back, then pivots to get the sideways view
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Grouped Rows" = Table.Group(Source, {"index", "product"}, {
{"data", each _, type table },
{"most_recent_target_sale", each Table.Sort(_,{{"time", Order.Descending}}){0}[target_sale], type number}
}),
#"Expanded data" = Table.ExpandTableColumn(#"Grouped Rows", "data", {"time", "sale"}, {"time", "sale"}),
#"Changed Type" = Table.TransformColumnTypes(#"Expanded data",{{"index", type number}}),
#"Pivoted Column" = Table.Pivot(#"Changed Type", List.Distinct(#"Changed Type"[time]), "time", "sale", List.Sum)
in #"Pivoted Column"

PowerBI - Calculating Daily Percent Returns and Cumulative Returns

I am trying to figure out a way of calculating/visualizing the cumulative percentage change in price (close price) of an ETF in PowerBI, such that I can create a line chart and filter the date range (i.e. select different periods with timeline slicer so I can show cumulative change over 1 month/year-to-date etc).
Effectively, I am trying to calculate the cumulative percent change in column 'close', however I have multiple ETFs (so column 'Symbol' has more than 1 ETF), and this means that there will need to be a different 'minimum close price' per ETF. I would like to be able to do this daily so I can produce a line chart to visualize the data.
I have tried doing this a few ways, am happy to provide details if helpful but me posting this sort of tells you how (un)successful I have been... Quite new to PowerBI/DAX so would really appreciate the help.
Here's one way of adding a column which groups by Symbol, and then adds a column which has the percent change (based on (currentClose-previousClose)/previousClose.
List.Generate generates the array of percentage change values
Add that List as a column to the table
Expand the grouped table
let
//Get table data from someplace
//edit next lines to reflect you actual source
Source = Excel.Workbook(File.Contents("C:\Users\ron\OneDrive\Documents\Book1.xlsx"), null, true),
Table21_Table = Source{[Item="Table21",Kind="Table"]}[Data],
//set the data types
#"Changed Type" = Table.TransformColumnTypes(Table21_Table,{
{"datetime", type date}, {"open", type number}, {"high", type any},
{"low", type any}, {"close", type number}, {"volume", type any}, {"Symbol", type text}}),
//Group rows by Symbol, then add a column with the percent change from close to close
#"Grouped Rows" = Table.Group(#"Changed Type", {"Symbol"}, {
{"Percent Change", (t) => Table.FromColumns(
Table.ToColumns(t) &
{List.Generate(
()=>[p=null, idx=0],
each [idx]< Table.RowCount(t),
each [p= (t[close]{[idx]+1}-t[close]{[idx]})/t[close]{[idx]}, idx=[idx]+1],
each [p])},type table [datetime=nullable date, open=nullable number, high=any, low=any, close=nullable number,
volume=any, Symbol=nullable text, percentChange=nullable number])
}
}),
//expand the created table and set the datatypes
#"Expanded Percent Change" = Table.ExpandTableColumn(#"Grouped Rows", "Percent Change", {"datetime", "open", "high", "low", "close", "volume", "percentChange"}, {"datetime", "open", "high", "low", "close", "volume", "percentChange"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Expanded Percent Change",{{"datetime", type date}, {"open", type number}, {"high", type any}, {"low", type any}, {"close", type number}, {"volume", type any}, {"percentChange", Percentage.Type}})
in
#"Changed Type1"

Convert wrapped table data to columns

I'm trying to do web scraping with power bi where I'm using the data from the following site:
https://pt.wikipedia.org/wiki/Jogo_do_bicho
After passing the site URL, the data came organized in the following format:
![Screenshot 1][1]
[1]: https://i.stack.imgur.com/HPjE7.png
where the number is an index related to the animal that has its specific thousand, how do I put everything organized in a column with all indices?
I have an example attached:
![Screenshot 2][2]
[2]: https://i.stack.imgur.com/cxWbU.png
I'll try to add detail later but I think this will work:
let
Source = Web.Page(Web.Contents("https://pt.wikipedia.org/wiki/Jogo_do_bicho")){0}[Data],
ToLists = List.Skip(Table.ToColumns(Source),1),
#"Converted to Table" = Table.FromList(ToLists, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Expanded Column1" = Table.ExpandListColumn(#"Converted to Table", "Column1"),
#"Added Custom" = Table.AddColumn(#"Expanded Column1", "Pivot", each if Text.Length([Column1]) = 2 then "Group" else "Animal"),
#"Added Index" = Table.AddIndexColumn(#"Added Custom", "Index", 0, 1),
#"Integer-Divided Column" = Table.TransformColumns(#"Added Index", {{"Index", each Number.IntegerDivide(_, 2), Int64.Type}}),
#"Pivoted Column" = Table.Pivot(#"Integer-Divided Column", List.Distinct(#"Integer-Divided Column"[Pivot]), "Pivot", "Column1"),
#"Split Column by Delimiter" = Table.SplitColumn(#"Pivoted Column", "Animal", Splitter.SplitTextByDelimiter("#(lf)#(cr)", QuoteStyle.Csv), {"Animal", "Values"}),
#"Trimmed Text" = Table.TransformColumns(#"Split Column by Delimiter",{{"Animal", Text.Trim, type text}, {"Values", Text.Trim, type text}}),
#"Changed Type" = Table.TransformColumnTypes(#"Trimmed Text",{{"Group", Int64.Type}}),
#"Removed Columns" = Table.RemoveColumns(#"Changed Type",{"Index"}),
#"Sorted Rows" = Table.Sort(#"Removed Columns",{{"Group", Order.Ascending}})
in
#"Sorted Rows"
Edit: The key here is to convert the table into a list of columns using Table.ToColumns. This turns it into a list of lists that we can convert into a table and expand into one long column.
Once all of the columns are stacked into one single column, we want to separate the group id from the details, which we can do in this case by checking the length of the text and defining a custom column that labels each row with a different data category.
With that categorization of the rows in place, we want to pivot that new custom column but we want an index column so it knows what stays together. Add an index column and integer divide by two so you get 0,0,1,1,2,2,3,3,... so that each pair gets its own unique ID. Now we can finally pivot.
Once pivoted, do any cleanup you feel like, e.g., splitting columns, trimming whitespace, changing column types, removing unneeded columns, and sorting.

SUMX over table variable breaks when adding relationships to underlying table

Environment
I have created the two queries in Power Query
"Demo"
let
Source = Table.FromList(List.Random(1000, 20200427),Splitter.SplitByNothing()),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
#"Split Column by Position" = Table.SplitColumn(#"Changed Type", "Column1", Splitter.SplitTextByPositions({0, 5, 7}, false), {"Month", "Amount"}),
#"Changed Type2" = Table.TransformColumnTypes(#"Split Column by Position",{{"Month", type number}, {"Amount", type number}}),
ConvetToMonthNumber = Table.TransformColumns(#"Changed Type2",{{"Month", each Number.RoundDown( 12 * _) + 1, Int64.Type}})
in
ConvetToMonthNumber
and "MockCal"
let
Source = Table.FromList( List.Repeat({1..4},3), Splitter.SplitByNothing(),{"CalQuart"}),
#"Added Index" = Table.AddIndexColumn(Source, "Month", 1, 1)
in
#"Added Index"
and then in the model I have two bits of DAX
Running Amount =
CALCULATE(
SUM('Demo'[Amount]),
FILTER(ALL('Demo'[Month]), 'Demo'[Month] <= MAX('Demo'[Month]))
)
and
WEIRD Run total =
VAR CalcTable = SUMMARIZE(Demo,Demo[Month],"MonthlyRollingAmounts",[Running Amount])
VAR TotalAmount = SUMX(CalcTable,[MonthlyRollingAmounts])
RETURN IF(ISFILTERED(Demo[Month]), [Running Amount], TotalAmount)
With this I can produce the following visual:
This achieves the desired result: Create a measure that sums up its slices by month.
However, when I create a relationship off this table, the SUMX stops working as expected.
The question is: Why does adding this relationship change the behavior of the SUMX? It doesn't seem to me like it should matter at all. Help is appreciated, this one is really bending my brain.
The reason for this is that in your Running Amount measure you remove any filtering on 'Demo'[Month] but this does not propagate upstream to MockCal (which I'm assuming you're using as the first column in your visual).
In general, you want to do your date filtering on your calendar table instead of your fact table. Try this instead:
Running Amount =
CALCULATE(
SUM('Demo'[Amount]),
FILTER(ALL(MockCal), 'MockCal'[Month] <= MAX('MockCal'[Month]))
)
Your WEIRD Run total should be updated as well to use the calendar table.

Power BI: Count multiple values in column

I'm working with the dataset where the gender of all participants is described in one single column. I'd like to create two new columns for both genders and fill it up with the number of males / females involved.
The syntax used in a column looks like this:
0::Male||1::Male||3::Male||4::Female
(so we have 4 participants, the value in col "Male" would be 3, in col "Female" 1)
Would you be so kind and help me to extract this information? ♥
I'm sorry to ask you as I know I'd be able to eventually find the solution by myself, but I'm really under pressure right now. :/
This is the screenshot of the column I wanna extract values from.
Thanks a lot to everyone who tries to help! :)
In powerquery, you'd need two splits (one on :: and one on ||, and then a pivot to get data into right format)
Select your data, including header column of participant_gender and load your table into powerquery with Data ... From Table/Range...[x] my data has headers. I assume below that this is the first table of your file, and is named Table1
Add Column..Index Column...
right click participant_gender column ... split column ... by delimiter ... custom ... || [x] each occurrence of the delimiter , Advanced options [x] rows
right click participant_gender column ... split column ... by delimiter ... custom ... :: [x] each occurrence of the delimiter (ignore advanced options)
Click to select both participant_gender.2 and Index columns .. right click group by ... group by (participant_gender.2) (index) new column name (count) operation (sum) column (participant_gender.1)
Click to select participant_Gender.2 column ... Transform..pivot column...Values column (Count)
File...Close and Load to ... table
Code produced, which you can paste into Home .. Advanced Editor... if you wish
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"participant_gender", type text}}),
#"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 0, 1),
#"Split Column by Delimiter" = Table.ExpandListColumn(Table.TransformColumns(#"Added Index", {{"participant_gender", Splitter.SplitTextByDelimiter("||", QuoteStyle.Csv), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "participant_gender"),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"participant_gender", type text}}),
#"Split Column by Delimiter1" = Table.SplitColumn(#"Changed Type1", "participant_gender", Splitter.SplitTextByDelimiter("::", QuoteStyle.Csv), {"participant_gender.1", "participant_gender.2"}),
#"Changed Type2" = Table.TransformColumnTypes(#"Split Column by Delimiter1",{{"participant_gender.1", Int64.Type}, {"participant_gender.2", type text}}),
#"Grouped Rows" = Table.Group(#"Changed Type2", {"participant_gender.2", "Index"}, {{"Count", each List.Sum([participant_gender.1]), type number}}),
#"Pivoted Column" = Table.Pivot(#"Grouped Rows", List.Distinct(#"Grouped Rows"[participant_gender.2]), "participant_gender.2", "Count", List.Sum)
in #"Pivoted Column"