EXPAND MULTIPLE COLUMNS POWER BI - powerbi

I´ve been struggling with this:
My table shows 3 records but when expanding there are like 100 columns. I used this code:
#"Expanded Data" = Table.ExpandTableColumn(#"Source", "Document", List.Union(List.Transform(#"Source"[Document]), each Table.ColumnNames(_))),
but it's not working. How can I expand simultaneously all columns? Also, inside those columns there are even more, for example I expand the first time end then those new columns have more records inside.
What could I do? Thanks in advance!

Try this ExpandAllRecords function - it recursively expands every Record-type column:
https://gist.github.com/Mike-Honey/0a252edf66c3c486b69b

This should work for Records Columns.
let
ExpandIt = (TableToExpand as table, optional ColumnName as text) =>
let
ListAllColumns = Table.ColumnNames(TableToExpand),
ColumnsTotal = Table.ColumnCount(TableToExpand),
CurrentColumnIndex = if (ColumnName = null) then 0 else List.PositionOf(ListAllColumns, ColumnName),
CurrentColumnName = ListAllColumns{CurrentColumnIndex},
CurrentColumnContent = Table.Column(TableToExpand, CurrentColumnName),
IsExpandable = if List.IsEmpty(List.Distinct(List.Select(CurrentColumnContent, each _ is record))) then false else true,
FieldsToExpand = if IsExpandable then Record.FieldNames(List.First(List.Select(CurrentColumnContent, each _ is record))) else {},
ColumnNewNames = List.Transform(FieldsToExpand, each CurrentColumnName &"."& _),
ExpandedTable = if IsExpandable then Table.ExpandRecordColumn(TableToExpand, CurrentColumnName, FieldsToExpand, ColumnNewNames) else TableToExpand,
NextColumnIndex = CurrentColumnIndex+1,
NextColumnName = ListAllColumns{NextColumnIndex},
OutputTable = if NextColumnIndex > ColumnsTotal-1 then ExpandedTable else #fx_ExpandIt(ExpandedTable, NextColumnName)
in
OutputTable
in
ExpandIt
This basically takes Table to Transform as the main argument,and then one by one checks if the Column Record is expandable (if column has "records" in it, it will expand it, otherwise move to next column and checks it again).
Then it returns the Output table once everything is expanded.
This function is calling the function from inside for each iteration.

Related

power bi DAX hierarchical table concatenation of names

Since two days I'm on a problem and I can't solve it so I come here to ask some help...
I have that bit of dax that basically take the path of a hierarchical table (integers) and take the string names of the 2 first in the path.
the names I use:
'HIERARCHY' the hierarchical table with names, id, path, nbrItems, string
mytable / addedcolumn1/2 the new table used to emulate the for loop
DisplayPath =
var __Path =PATH(ParentChild[id], ParentChild[parent_id])
var __P1 = PATHITEM(__Path,1) var __P2 = PATHITEM(__Path,2)
var l1 = LOOKUPVALUE(ParentChild[Place],ParentChild[id],VALUE(__P1))
var l2a = LOOKUPVALUE(ParentChild[Place],ParentChild[id],VALUE(__P2))
var l2 = if(ISBLANK(l2a), "", " -> " & l2a)
return CONCATENATE(l1,l2)
My problem is... I don't know the number of indexes in my path, can go from 0 to I guess 15...
I've tried some things but can't figure out a solution.
First I added a new column called nbrItems which calculate the number of items in the list of the path.
The two columns:
Then I added that bit of code that emulates a for loop depending on the number of items in the path list, and I'd like in it to
get name of parameters
concatenate them in one string that I can return and get
string =
var n = 'HIERARCHY'[nbrItems]
var mytable = GENERATESERIES(1, n)
var addedcolumn1 = ADDCOLUMNS(mytable, "nom", /* missing part: get name */)
var addedcolumn2 = ADDCOLUMNS(addedcolumn1, "string", /* missing part: concatenate previous concatenated and new name */)
var mymax = MAXX(addedcolumn2, [Value])
RETURN MAXX(FILTER(addedcolumn2, [Value] = mymax), [string])
Full table:
Thanks for your help in advance!
Ok, so after some research and a lot of try and error... I've came up to a nice and simple solution:
The original problem was that I had a hierarchical table ,but with all data in the same table.
like so
What I did was, adding a new "parent" column with this dax:
parent =
var a = 'HIERARCHY'[id_parent]
var b = CALCULATE(MIN('HIERARCHY'[libelle]), FILTER(ALL('HIERARCHY'), 'HIERARCHY'[id_h] = a))
RETURN b
This gets the parent name from the id_parent (ref. screen).
then I could just use the path function, not on the id's but on the names... like so:
path = PATH('HIERARCHY'[libelle], 'HIERARCHY'[parent])
It made the problem easy because I didn't need to replace the id's by there names after this...
and finally to make it look nice, I used some substitution to remove the pipes:
formated_path = SUBSTITUTE('HIERARCHY'[path], "|", " -> ")
final result

Custom Function Power Query (M) - Return Table

I need a custom function that takes two parameters, Column1 and Column2, so:
For each Row, return the value of Column1 but only if exists a Value in the Column2 else return null
I have tried this:
let ColumnsFilter = (Tabla,C1,C2)=>
Table.AddColumn(Tabla, "Custom", each if [C2] <> null then [C1] else null)
in
ColumnsFilter
And calling the function:
#"Previous Step" = .....
#"P" = ColumnsFilter(#"Previous Step","Column1","Column2")
in
P
And is not working. clearly I am not using the syntax properly.
In summary I need a table as input and a table as output adding custom columns.
How can I write this?
(Please don't tell me to use the assisted of Power Query, I need to write similar functions manually)
Since you're passing column names as text and individual rows are a record type, you have to use Record.Field to pull the right column (field) from the current row (record).
let
ColumnsFilter = (Tabla as table, C1 as text, C2 as text) as table =>
Table.AddColumn(Tabla, "Custom",
each if Record.Field(_, C2) <> null then Record.Field(_, C1) else null
)
in
ColumnsFilter

Convert a table into a function that can act like a Table.SelectRows condition

I have a table of Project:
that I would like to filter by the FIELD, OPERATOR, and VALUE columns contained in the Project Group table:
The Power Query M to apply this filter would be:
let
Source = #"Project",
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Projectid", Int64.Type}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each [Projectid] >= 100000 and [Projectid] <= 500000)
in
#"Filtered Rows"
Results (need to remove the error row):
How do I convert the FIELD, OPERATOR, and VALUE columns into a function that can be used as a condition for the SelectRows function?
If you need to do comparisons, might be best to first change the types of the columns (in both tables) that are being compared. Preferably to type number.
The code below assumes that:
the OPERATOR column of Project Group table can only contain: > or < and that these values should be interpreted as >= and <= respectively.
the column in Project table (that needs to be compared) can change and its name will be in the FIELD column of the Project Group. It's assumed that the name matches exactly. If this is not the case, you might need to standardise things (or at least perform a case-insensitive search) to ensure values can be mapped to column names correctly.
Based on the assumptions above, here's one approach:
let
// Dummy table for example purposes
project = Table.FromColumns({
{0..10},
{5..15}
}, type table [projectId = number, name = number]),
// Dummy table for example purposes
projectGroup = Table.FromColumns({
{"projectId", "projectId"},
{">", "<"},
{5, 7}
}, type table [FIELD = text, OPERATOR = text, VALUE = number]),
// Should take in a row from "Project" table and return a boolean
// representing whether said row matches the criteria contained
// within "Project Group" table.
selectorFunc = (projectRow as record) as logical =>
let
shouldKeepProjectRow = Table.MatchesAllRows(projectGroup, (projectGroupRow as record) =>
let
fieldNameToCheck = projectGroupRow[FIELD],
valueFromProjectRow = Record.Field(projectRow, fieldNameToCheck),
compared = if projectGroupRow[OPERATOR] = ">" then
valueFromProjectRow >= projectGroupRow[VALUE]
else
valueFromProjectRow <= projectGroupRow[VALUE]
in compared
)
in shouldKeepProjectRow,
selectedRows = Table.SelectRows(project, selectorFunc)
in
selectedRows
The main function used is Table.MatchesAllRows (https://learn.microsoft.com/en-us/powerquery-m/table-matchesallrows).
Another approach could potentially be: Expression.Evaluate: https://learn.microsoft.com/en-us/powerquery-m/expression-evaluate. However, I've not used it, so I'm not sure whether there are any "gotchas"/implications to be aware of.

Skip a record if empty

I've created a function that cleans my data of extra columns with null values. There should always be 15 columns after this however occasionally there is more or less and when this happens those tables should just be removed.
I've tried just skipping all those rows and returning an empty table but when I try to expand those tables I get an error "Cannot convert the value false to type Number."
(tbl as table) =>
let
ColumnNames = Table.ColumnNames(tbl),
RemoveNullColumns = Table.SelectColumns(tbl, List.Select(ColumnNames, each List.MatchesAny(Table.Column(tbl, _), each _ <> null))),
CheckColumns = Table.Skip(RemoveNullColumns, Table.ColumnCount(RemoveNullColumns) <> 15)
in
CheckColumns
See if this works for you. Removes any columns containing a null and returns tbl only if there are 15 remaining columns
(tbl as table) =>
let ColumnNames = Table.ColumnNames(tbl),
ReplacedValue = Table.ReplaceValue(tbl,null,"imanull",Replacer.ReplaceValue,ColumnNames ),
UnpivotedColumns = Table.UnpivotOtherColumns(ReplacedValue, {}, "Attribute", "Value"),
FilteredRows = Table.SelectRows(UnpivotedColumns, each ([Value] = "imanull")),
NonNullColumns= List.Difference(ColumnNames,List.Distinct(FilteredRows[Attribute])),
Results = if List.Count (NonNullColumns) <> 15 then null else Table.SelectColumns(tbl,NonNullColumns)
in Results

Power Query conditional sumif

I need to add a column that Sums the value column of all columns that have a common id. However, any id = null is not summed, but equal to the value column.
The above example should result in:
TopPaymendId JournalLineNetAmount TopAmount
fcbcd407-ca26-4ea0-839a-c39767d05403 -3623.98 -7061.23
fcbcd407-ca26-4ea0-839a-c39767d05403 -3437.25 -7061.23
ce77faac-1638-40e9-ad62-be1813ce9031 -88.68 -88.68
531d9bde-3f52-47f3-a9cf-6f3566733af2 -152.23 -152.23
8266dfef-dd14-4654-a6d2-091729defde7 229.42 229.42
f8b97a47-15ef-427d-95e0-ce23cc8efb1f -777 -777
null -3.01 -3.01
null -2.94 -2.94
null 3312.5 3312.5
This code should work:
let
Source = Excel.CurrentWorkbook(){[Name="Data"]}[Content],
group = Table.Group(Source, {"TopPaymendId"}, {"TopAmount", each List.Sum([JournalLineNetAmount])}),
join = Table.Join(Source,{"TopPaymendId"},group,{"TopPaymendId"}),
replace = Table.ReplaceValue(join,each [TopAmount],each if [TopPaymendId] = null
then [JournalLineNetAmount] else [TopAmount],Replacer.ReplaceValue,{"TopAmount"})
in
replace