Power Query: recursive function to append elements in a table - powerbi

I am trying to use a recursive function to append values from a list to a table, however the code below only show me the first and second results:
let
Source = {"second", "third", "forth", "fith", "seventh", "eighth"},
Count = List.Count(Source),
Table = Table.FromRecords({[sequence = "first"]}, type table[sequence = text]),
appendTbl = (x as list, n as number, tbl as table) =>
let
appTable = Table.InsertRows(Table, n, {[sequence = Source{n}]}),
Check = if n = (Count-1) then #appendTbl(x, n+1, appTable) else appTable
in
Check,
Result = appendTbl(Source, 0, Table)
in
Result
Can anyone please give me a help? Thanks !

Its kind of hard to tell if you are using the number to designate the spot in the table you want to insert, or the number of times you want to duplicate the array before inserting it into the table
That said, you can combine tables with Table.Combine() after converting the list to a table with Table.FromList(). If you need to append it multiple times then just use List.Repeat on the list. If you need to use the Count variable in your function, you have to send it there appendTbl = (x as list, n as number, tbl as table, count as number) =>
some sample codes that probably don't do exactly what you want
let Source = {"second", "third", "forth", "fith", "seventh", "eighth"},
AppendCount=2, //# times to append the list onto the table
#"Converted to Table" = Table.FromList(List.Repeat(Source,AppendCount), Splitter.SplitByNothing(), null),
#"Renamed Columns" = Table.RenameColumns(#"Converted to Table",{{"Column1", "sequence"}}),
Table = Table.FromRecords({[sequence = "first"]}, type table[sequence = text]),
combined= Table.Combine({Table, #"Renamed Columns"})
in combined
or
let Source = {"second", "third", "forth", "fith", "seventh", "eighth"},
Table = Table.FromRecords({[sequence = "first"]}, type table[sequence = text]),
appendTbl = (x as list, n as number, tbl as table) => // append list x to table tbl, n times on column sequence
let #"Converted to Table" = Table.FromList(List.Repeat(x,n), Splitter.SplitByNothing(), null),
#"Renamed Columns" = Table.RenameColumns(#"Converted to Table",{{"Column1", "sequence"}}),
combined= Table.Combine({tbl, #"Renamed Columns"})
in combined,
Result = appendTbl(Source, 2, Table) // append Source to Table, 2 times
in Result

Related

generate a one-column table that contains hundreds of different categories using M or DAX

I need to split my products into a total of 120 predefined price clusters/buckets. These clusters can overlap and look somewhat like that:
As I dont want to write down all of these strings manually: Is there a convenient way to do this in M or DAX directly using a bit of code?
Thanks in advance!
Dave
With m-Query you can create a function. Open the query editor. Richt click and create empty query. Create function (ignore warning) and call it : RowGenerator.
Open advanced editor and past the following code:
let
Bron = (base as number, start as number, end as number) => let
Bron = Table.FromList(List.Generate(() => start, each _ <= end, each _ + 1), Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Aangepaste kolom toegevoegd" = Table.AddColumn(Bron, "Aangepast", each Number.ToText(base) & " - " & Number.ToText([Column1]))
in
#"Aangepaste kolom toegevoegd"
in
Bron
This function creates a table where base is your first number and start, end the range.
Add another empty query, open the advanged editor and paste:
let
Bron = List.Generate(() => 0, each _ < 5, each _ + 1),
#"Geconverteerd naar tabel" = Table.FromList(Bron, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Aangeroepen aangepaste functie" = Table.AddColumn(#"Geconverteerd naar tabel", "test", each RowGenerator(_[Column1], _[Column1] + 1, 5)),
#"test uitgevouwen" = Table.ExpandTableColumn(#"Aangeroepen aangepaste functie", "test", {"Column1", "Aangepast"}, {"Column1.1", "Price Cluster"}),
#"Kolommen verwijderd" = Table.RemoveColumns(#"test uitgevouwen",{"Column1", "Column1.1"})
in
#"Kolommen verwijderd"
This creates first a list of 5 rows, then it calls the previous made function for each row and the last step is to expend the rows and remove the not needed columns.
Enjoy:
You can create this bucket by DAX (New Table):
Table = SELECTCOLUMNS(
GENERATE(SELECTCOLUMNS(GENERATESERIES(0,10,1),"FirstPart",[Value]), SELECTCOLUMNS(GENERATESERIES(0,10,1),"SecondPart",[Value]))
,"Bucket", [FirstPart] & " - " & [SecondPart]
)
Table = SELECTCOLUMNS(
GENERATE(SELECTCOLUMNS(GENERATESERIES(0,9,1),"FirstPart",[Value]), TOPN([FirstPart], SELECTCOLUMNS(GENERATESERIES(1,9,1),"SecondPart",[Value]), [SecondPart],ASC))
,"Bucket", [FirstPart] & " - " & [SecondPart]
)

Convert a table into a function that can act like a Table.SelectRows condition

I have a table of Project:
that I would like to filter by the FIELD, OPERATOR, and VALUE columns contained in the Project Group table:
The Power Query M to apply this filter would be:
let
Source = #"Project",
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Projectid", Int64.Type}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each [Projectid] >= 100000 and [Projectid] <= 500000)
in
#"Filtered Rows"
Results (need to remove the error row):
How do I convert the FIELD, OPERATOR, and VALUE columns into a function that can be used as a condition for the SelectRows function?
If you need to do comparisons, might be best to first change the types of the columns (in both tables) that are being compared. Preferably to type number.
The code below assumes that:
the OPERATOR column of Project Group table can only contain: > or < and that these values should be interpreted as >= and <= respectively.
the column in Project table (that needs to be compared) can change and its name will be in the FIELD column of the Project Group. It's assumed that the name matches exactly. If this is not the case, you might need to standardise things (or at least perform a case-insensitive search) to ensure values can be mapped to column names correctly.
Based on the assumptions above, here's one approach:
let
// Dummy table for example purposes
project = Table.FromColumns({
{0..10},
{5..15}
}, type table [projectId = number, name = number]),
// Dummy table for example purposes
projectGroup = Table.FromColumns({
{"projectId", "projectId"},
{">", "<"},
{5, 7}
}, type table [FIELD = text, OPERATOR = text, VALUE = number]),
// Should take in a row from "Project" table and return a boolean
// representing whether said row matches the criteria contained
// within "Project Group" table.
selectorFunc = (projectRow as record) as logical =>
let
shouldKeepProjectRow = Table.MatchesAllRows(projectGroup, (projectGroupRow as record) =>
let
fieldNameToCheck = projectGroupRow[FIELD],
valueFromProjectRow = Record.Field(projectRow, fieldNameToCheck),
compared = if projectGroupRow[OPERATOR] = ">" then
valueFromProjectRow >= projectGroupRow[VALUE]
else
valueFromProjectRow <= projectGroupRow[VALUE]
in compared
)
in shouldKeepProjectRow,
selectedRows = Table.SelectRows(project, selectorFunc)
in
selectedRows
The main function used is Table.MatchesAllRows (https://learn.microsoft.com/en-us/powerquery-m/table-matchesallrows).
Another approach could potentially be: Expression.Evaluate: https://learn.microsoft.com/en-us/powerquery-m/expression-evaluate. However, I've not used it, so I'm not sure whether there are any "gotchas"/implications to be aware of.

Multiple row into one row

I'm new in powerbi and i'm looking some help with a transformation.
What i'm trying to do with powerquery :
first i want to group the following columns : call_key ivr_agent cli dnis lang_id
and after i need to copy the other infos into one row only : all other info need to go on one row.
the second row with same call_key (and others) need to go on a new column.
In few words:
I need that all rows with same call_key are on one row only
File excel test : https://1drv.ms/x/s!AqE6W5akVSvUh59KfGmUiCSnZH6OVg
Thank you so much for your help,
Phil
I couldn't understand exactly if you needed the rows in new columns or just merge them in a single one.
For merging in a single one, try this query:
let
Origen = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
RemoveBlanks = Table.SelectRows(Origen, each [call_key] <> null and [call_key] <> ""),
CombineCols = Table.AddColumn(RemoveBlanks, "MergedCol", each Text.Combine({Text.From([action_time], "es-CO"), [ivr_module], [action_location], [action_type], [action], [action_data1_desc], Text.From([action_data1_value], "es-CO"), [action_data2_desc], [action_data2_value], [action_data3_desc], Text.From([action_data3_value], "es-CO")}, "|"), type text),
RemoveCols = Table.SelectColumns(CombineCols,{"call_key", "ivr_agent", "cli", "dnis", "lang_id", "MergedCol"}),
GroupAndMerge = Table.Group(RemoveCols, {"call_key", "ivr_agent", "cli", "dnis", "lang_id"}, {{"New", each Text.Combine([MergedCol], "#(lf)"), type text}})
in
GroupAndMerge
EDIT: You may split it again, like this:
let
Origen = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
RemoveBlanks = Table.SelectRows(Origen, each [call_key] <> null and [call_key] <> ""),
CombineCols = Table.AddColumn(RemoveBlanks, "MergedCol", each Text.Combine({Text.From([action_time], "es-CO"), [ivr_module], [action_location], [action_type], [action], [action_data1_desc], Text.From([action_data1_value], "es-CO"), [action_data2_desc], [action_data2_value], [action_data3_desc], Text.From([action_data3_value], "es-CO")}, "|"), type text),
RemoveCols = Table.SelectColumns(CombineCols,{"call_key", "ivr_agent", "cli", "dnis", "lang_id", "MergedCol"}),
GroupAndMerge = Table.Group(RemoveCols, {"call_key", "ivr_agent", "cli", "dnis", "lang_id"}, {{"New", each Text.Combine([MergedCol], "#(lf)"), type text}}),
SplitColumn = Table.SplitColumn(GroupAndMerge, "New", Splitter.SplitTextByDelimiter("|", QuoteStyle.Csv), {"New.1", "New.2", "New.3", "New.4", "New.5", "New.6", "New.7", "New.8", "New.9", "New.10", "New.11", "New.12", "New.13", "New.14", "New.15", "New.16", "New.17", "New.18", "New.19", "New.20", "New.21", "New.22", "New.23", "New.24", "New.25", "New.26", "New.27", "New.28", "New.29", "New.30", "New.31", "New.32", "New.33", "New.34", "New.35", "New.36", "New.37", "New.38", "New.39", "New.40", "New.41", "New.42", "New.43", "New.44", "New.45", "New.46", "New.47", "New.48", "New.49", "New.50", "New.51", "New.52", "New.53", "New.54", "New.55", "New.56", "New.57", "New.58", "New.59"})
in
SplitColumn

Skip a record if empty

I've created a function that cleans my data of extra columns with null values. There should always be 15 columns after this however occasionally there is more or less and when this happens those tables should just be removed.
I've tried just skipping all those rows and returning an empty table but when I try to expand those tables I get an error "Cannot convert the value false to type Number."
(tbl as table) =>
let
ColumnNames = Table.ColumnNames(tbl),
RemoveNullColumns = Table.SelectColumns(tbl, List.Select(ColumnNames, each List.MatchesAny(Table.Column(tbl, _), each _ <> null))),
CheckColumns = Table.Skip(RemoveNullColumns, Table.ColumnCount(RemoveNullColumns) <> 15)
in
CheckColumns
See if this works for you. Removes any columns containing a null and returns tbl only if there are 15 remaining columns
(tbl as table) =>
let ColumnNames = Table.ColumnNames(tbl),
ReplacedValue = Table.ReplaceValue(tbl,null,"imanull",Replacer.ReplaceValue,ColumnNames ),
UnpivotedColumns = Table.UnpivotOtherColumns(ReplacedValue, {}, "Attribute", "Value"),
FilteredRows = Table.SelectRows(UnpivotedColumns, each ([Value] = "imanull")),
NonNullColumns= List.Difference(ColumnNames,List.Distinct(FilteredRows[Attribute])),
Results = if List.Count (NonNullColumns) <> 15 then null else Table.SelectColumns(tbl,NonNullColumns)
in Results

Converting non structured key value pairs data to a table

I have data in the following format, sample shown below:
ValA=101
ValB=2938
ValA=998
ValB=387
ValA=876
ValB=9832
I know that each set of ValA & ValB are a set of values that belong together, so output will be:
ValA ValB
101 2938
998 387
.......
.......
I need to get this into a tabular format so each valA ValB pair is one row.
Ive tried doing this in powerquery by splitting on the = sign and then pivoting on the Val name, but it doesnt work.
any idea on how this might be easily achieved in powerquery?
Thanks!
I ended up doing the exact same as Lukasz, here's the full code:
let
Source = "ValA=101
ValB=2938
ValA=998
ValB=387
ValA=876
ValB=9832",
Custom1 = Lines.FromText(Source),
#"Converted to Table" = Table.FromList(Custom1, Splitter.SplitTextByDelimiter("="), null, null, ExtraValues.Error),
ChangedType = Table.TransformColumnTypes(#"Converted to Table",{{"Column1", type text}, {"Column2", Int64.Type}}),
CustomA = Table.AddColumn(ChangedType, "ValA", each if [Column1] = "ValA" then [Column2] else null),
CustomB = Table.AddColumn(CustomA, "ValB", each if [Column1] = "ValB" then [Column2] else null),
FilledDown = Table.FillDown(CustomB,{"ValA"}),
FilteredRows = Table.SelectRows(FilledDown, each [ValB] <> null)
in
FilteredRows
Lukasz's second idea using pivot columns looks like this:
let
Source = "ValA=101
ValB=2938
ValA=998
ValB=387
ValA=876
ValB=9832",
Custom1 = Lines.FromText(Source),
#"Converted to Table" = Table.FromList(Custom1, Splitter.SplitTextByDelimiter("="), null, null, ExtraValues.Error),
ChangedType = Table.TransformColumnTypes(#"Converted to Table",{{"Column1", type text}, {"Column2", Int64.Type}}),
AddedIndex = Table.AddIndexColumn(ChangedType, "Index", 0, 1),
IntegerDividedColumn = Table.TransformColumns(AddedIndex, {{"Index", each Number.IntegerDivide(_, 2), Int64.Type}}),
PivotedColumn = Table.Pivot(IntegerDividedColumn, List.Distinct(IntegerDividedColumn[Column1]), "Column1", "Column2")
in
PivotedColumn
The trick I found was to add divided-by-two index column (that goes 0, 0, 1, 1, 2, 2...) so the pivot knows the first two rows should be related, and the next two, etc.
You can do the following:
1) create two new calculated columns with logic like if column1 contains ValA then Column1 else null. same logic for ValB in second column.
2) use the fill down feature on the left most column. This will produce rows with values for both ValA and ValB in distinct columns
3) use the filter feature to filter out rows that have nulls in your two new columns
That should give you what you want.
Edit: thinking about this more you might also try: split column1 on the equal sign. Then pivot the new column and it should produce two columns with the discrete values. HTH.