I would like to compare 2 consecutive rows of the same column ("Niveau num") of a table in PBI and display the result of this comparison in another column ("Niveau arbre"). This can be done very easily in Excel but Power BI is online and it is more complicated to set it up.
I added an index column to have access to the row number and I wrote the following formula in Power Query:
= Table.AddColumn(#"Colonnes permutées", "Niveau arbre", each if Estimations[Niveau num]{[Index]} > Estimations[Niveau num]{[Index]+1} then 1 else 0, type number)
screenshot
I then get the error:
Expression.Error : A cyclic reference was detected during the evaluation.
Here is all the code:
let
Source = Access.Database(File.Contents("C:\Users\T0275244\Desktop\BDD Access PBI\Off\BDD-Off.accdb"), [CreateNavigationProperties=true]),
_Estimations = Source{[Schema="",Item="Estimations"]}[Data],
#"Personnalisée ajoutée" = Table.AddColumn(_Estimations, "Niveau num", each List.Sum(List.Transform(Text.ToList([WBS]), each if _ = "." then 0 else 1))),
#"Lignes filtrées" = Table.SelectRows(#"Personnalisée ajoutée", each [WBS] <> null and [WBS] <> ""),
#"Type modifié" = Table.TransformColumnTypes(#"Lignes filtrées",{{"Niveau num", Int64.Type}}),
#"Index ajouté" = Table.AddIndexColumn(#"Type modifié", "Index", 1, 1, Int64.Type),
#"Colonnes permutées" = Table.ReorderColumns(#"Index ajouté",{"Devis", "WBS", "Désignation", "Qté", "Respde Lot niveau2", "Centre de taux", "Activity", "Activities", "ECPS/Transverse", "SW Comp", "MO ING", "MO TECH", "Total MO (H)", "Total MO (K€)", "Autres MO", "Total MO et Autre MO", "Total tarifs (PRV)", "Divers CPP", "Total MO et Div Tarifs", "Durée ID Ingé", "Nb de Voyages", "Divers transports", "Total Frais Déplacement", "INTRAGROUP PURCHASES", "EXTERNAL PURCHASES", "Total DEVIS (non inflaté)", "Commentaires", "Options", "Devis(Devis)", "Index", "Niveau num"}),
#"Personnalisée ajoutée1" = Table.AddColumn(#"Colonnes permutées", "Niveau arbre", each if Estimations[Niveau num]{[Index]} > Estimations[Niveau num]{[Index]+1} then 1 else 0, type number)
in
#"Personnalisée ajoutée1"
Would you have an idea to solve this?
Thanks in advance for your help
change
#"Personnalisée ajoutée1" = Table.AddColumn(#"Colonnes permutées", "Niveau arbre", each if Estimations[Niveau num]{[Index]} > Estimations[Niveau num]{[Index]+1} then 1 else 0, type number)
to
#"Personnalisée ajoutée1" = Table.AddColumn(#"Colonnes permutées", "Niveau arbre", each if [Niveau num]{[Index]} > [Niveau num]{[Index]+1} then 1 else 0, type number)
you cannot refer to Estimations which is (a) the wrong name, since you used _Estimations, and (b) wrong since that step would not have the index added yet anyway
EDIT -- from your updated image, your column is not numerical so you can't use >. Change the column format to numerical before the last Table.AddColumn()
FINAL EDIT
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", Int64.Type}}),
#"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 0, 1, Int64.Type),
#"Added Custom" = Table.AddColumn(#"Added Index", "Custom", each try if #"Added Index"{[Index]}[Column1] > #"Added Index"{[Index]+1}[Column1] then 1 else 0 otherwise 0)
in #"Added Custom"
Related
I'm trying to find the frequency of combinations that occur per an ID value.
Example given here: https://i.stack.imgur.com/ZG9gJ.png
The problem is that the number of rows that could make up a combination is variable, meaning a combination could consist of just 1 value or 2, 3, 4, etc.
I'm currently trying to do this within Power BI, but perhaps another tool would be more appropriate.
You can do this with Power Query (from Power BI => Transform)
Basic algorithm
Group by ID
for each subGroup
Concatenate a sorted list of Cats
Count the number of Cats per ID for subsequent sorting
Then Group by COMBI
Aggregate with Count function
M Code
let
//change next line to however you are getting your table
Source = Excel.CurrentWorkbook(){[Name="idCat"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}, {"Cat1", type text}}),
//group by ID and create COMBI
// and also length of each cat string for subsequen intelligent sorting
#"Grouped Rows" = Table.Group(#"Changed Type", {"ID"}, {
{"COMBI", each Text.Combine(List.Sort([Cat1])),type text},
{"lenCat", each Text.Length(Text.Combine(List.Sort([Cat1]))),Int64.Type}
}),
maxLen = List.Max(#"Grouped Rows"[lenCat]),
#"Delete length column" = Table.RemoveColumns(#"Grouped Rows","lenCat"),
//Group by Cats for counting
#"Grouped Cats" = Table.Group(#"Delete length column",{"COMBI"},{
{"COUNT", each List.Count([COMBI]), Int64.Type}
}),
#"Pad COMBI for Sorting" = Table.TransformColumns(#"Grouped Cats",{"COMBI", each Text.PadStart(_,maxLen), type text}),
#"Sorted Rows" = Table.Sort(#"Pad COMBI for Sorting",{{"COMBI", Order.Ascending}}),
#"Trim Leading Spaces" = Table.TransformColumns(#"Sorted Rows",{"COMBI", each Text.Trim(_), type text})
in
#"Trim Leading Spaces"
In Power BI, how does one count the unique values in a column and replace those values with the value name and count total? Doing things this way seems like the only way to get the totals inside of a visual's legend.
Starting data example:
Type
Example1
Example1
Example2
Example3
Example2
Example5
Example3
Example4
Example1
Example5
Example1
Example1
Example2
Example3
Example2
Example5
Example3
Example4
Example1
Example5
Example1
Desired result:
Type
Example1 (7)
Example1 (7)
Example2 (4)
Example3 (4)
Example2 (4)
Example5 (4)
Example3 (4)
Example4 (2)
Example1 (7)
Example5 (4)
Example1 (7)
Example1 (7)
Example2 (4)
Example3 (4)
Example2 (4)
Example5 (4)
Example3 (4)
Example4 (2)
Example1 (7)
Example5 (4)
Example1 (7)
You could use Powerquery for it like that.
In the example I took an input table from Excel which means you have to adjust Source accordingly
let
Source = Excel.CurrentWorkbook(){[Name="Tabelle1"]}[Content],
#"Hinzugefügter Index" = Table.AddIndexColumn(Source, "Index", 1, 1, Int64.Type),
groupedbyType = Table.Group(#"Hinzugefügter Index", {"Type"}, {{"CountIt", each _, type table [Type=text, Index=number]}}),
addCol = Table.AddColumn(groupedbyType, "Count", each List.Count([CountIt][Type])),
#"Erweiterte CountIt" = Table.ExpandTableColumn(addCol, "CountIt", {"Index"}, {"CountIt.Index"}),
#"Hinzugefügte benutzerdefinierte Spalte" = Table.AddColumn(#"Erweiterte CountIt", "Result", each [Type] & " (" & Text.From([Count]) & ")"),
#"Sortierte Zeilen" = Table.Sort(#"Hinzugefügte benutzerdefinierte Spalte",{{"CountIt.Index", Order.Ascending}}),
#"Entfernte Spalten" = Table.RemoveColumns(#"Sortierte Zeilen",{"Type", "CountIt.Index", "Count"})
in
#"Entfernte Spalten"
First step in Power Query looks like
Last step looks like
What I am doing is that
I add an index to restore the sorting later
I group by Type but keep the rows
I count the number of rows per group
I extend the result
I combine Type and the result of the count as requested
I restore the sorting
I remove all other columns
Update: Below same steps with translated steps
let
Source = Excel.CurrentWorkbook(){[Name="Tabelle1"]}[Content],
addedIndex = Table.AddIndexColumn(Source, "Index", 1, 1, Int64.Type),
groupedbyType = Table.Group(addedIndex, {"Type"}, {{"CountIt", each _, type table [Type=text, Index=number]}}),
addCol = Table.AddColumn(groupedbyType, "Count", each Table.RowCount([CountIt])),
extendCountIt = Table.ExpandTableColumn(addCol, "CountIt", {"Index"}, {"CountIt.Index"}),
addedColResult = Table.AddColumn(extendCountIt, "Result", each [Type] & " (" & Text.From([Count]) & ")"),
sortAgain = Table.Sort(addedColResult,{{"CountIt.Index", Order.Ascending}}),
removedCol = Table.RemoveColumns(sortAgain,{"Type", "CountIt.Index", "Count"})
in
removedCol
Update for Power BI: For Power BI the M-Code could look like that where an Excel file is used as data source.
let
Source = Excel.Workbook(File.Contents("Excelfilename.xlsx"), null, true),
Table1 = Source{[Item="Tabelle1",Kind="Table"]}[Data],
addedIndex = Table.AddIndexColumn(Table1, "Index", 1, 1, Int64.Type),
groupedbyType = Table.Group(addedIndex, {"Type"}, {{"CountIt", each _, type table [Type=text, Index=number]}}),
addCol = Table.AddColumn(groupedbyType, "Count", each Table.RowCount([CountIt])),
extendCountIt = Table.ExpandTableColumn(addCol, "CountIt", {"Index"}, {"CountIt.Index"}),
addedColResult = Table.AddColumn(extendCountIt, "Result", each [Type] & " (" & Text.From([Count]) & ")"),
sortAgain = Table.Sort(addedColResult,{{"CountIt.Index", Order.Ascending}}),
removedCol = Table.RemoveColumns(sortAgain,{"Type", "CountIt.Index", "Count"})
in
removedCol
Alternate method
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Added Custom" = Table.AddColumn(Source,"count",(i)=>Table.RowCount(Table.SelectRows(Source, each [Type]=i[Type])), type number),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom", each [Type] & " (" & Text.From([count]) & ")"),
#"Removed Other Columns" = Table.SelectColumns(#"Added Custom1",{"Custom"})
in #"Removed Other Columns"
I created this API paginated below. It roughly works, but in the OFFSET property, I need to stipulate instead of the number of the next sequence number of the record, for example, for the second page, the number 251, the next record of the second page, and so on.
My record limit per page is 250
The field totalItems returned the total of records, for example: 4500
I divide the total number of records by the total number of records per page, to get to know how many pages my API has: pageRange = {0..Number.RoundUp(totalItems / 250)}
When going to the second page, what happens in the API below, is that the records of the second page are coming repeated, because I should instead use the number 1 (referring to the second page), pass the number 251, and then, when doing the loop again, pass the number 501, until finishing the whole sequence (this parameter in the API is: offset=).
I need alter this line to include the FOR/WHILE for the item "ufnCallAPI(_)" of pages = List.Transform(pageRange, each ufnCallAPI(_)),
For example, the item above:
List.Transform(pageRange, each ufnCallAPI(_)),
List.Transform(pageRange, each ufnCallAPI(250)),
List.Transform(pageRange, each ufnCallAPI(500)),
List.Transform(pageRange, each ufnCallAPI(750)),
up to the total number totalItems
and include a FOR/WHILE to modified my API to not pass the number of the next page, but the number of the beginning of the list of the next item start (offset).
Thanks very much!
My code:
let
ufnCallAPI = (offSet) =>
let
query = Web.Contents("https://api.vhsys.com/v2/pedidos?offset=" & Number.ToText(offSet) & "&limit=250",
[Headers=[#"access-token"="OCKNYbAMaDgLBZBSQPCOGPWOXGSbdO", #"secret-access-token"="XXXXXXXXXXXXXX"]]),
result = Json.Document(query)
in
result,
tmpResult = ufnCallAPI(1),
auxTotal1 = Record.ToTable(tmpResult),
Value = auxTotal1{2}[Value],
auxTotal2 = Value[total],
totalItems = auxTotal2 -1,
pageRange = {0..Number.RoundUp(totalItems / 250)},
pages =List.Transform(pageRange, each ufnCallAPI(_)),
pages2 = Table.FromList(pages, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
pages3 = Table.ExpandRecordColumn(pages2, "Column1", {"code", "status", "paging", "data"}, {"Column1.code", "Column1.status", "Column1.paging", "Column1.data"}),
pages4 = Table.ExpandListColumn(pages3, "Column1.data"),
pages5 = Table.RemoveColumns(pages4,{"Column1.code", "Column1.status", "Column1.paging"}),
data = Table.ExpandRecordColumn(pages5, "Column1.data", {"id_ped", "id_pedido", "id_cliente", "nome_cliente", "id_local_retirada", "id_local_cobranca", "vendedor_pedido", "vendedor_pedido_id", "listapreco_produtos", "valor_total_produtos", "desconto_pedido", "desconto_pedido_porc", "peso_total_nota", "peso_total_nota_liq", "frete_pedido", "valor_total_nota", "valor_baseICMS", "valor_ICMS", "valor_baseST", "valor_ST", "valor_IPI", "condicao_pagamento_id", "condicao_pagamento", "frete_por_pedido", "transportadora_pedido", "id_transportadora", "data_pedido", "prazo_entrega", "referencia_pedido", "obs_pedido", "obs_interno_pedido", "status_pedido", "contas_pedido", "comissao_pedido", "estoque_pedido", "ordemc_emitido", "data_cad_pedido", "data_mod_pedido", "id_aplicativo", "id_pedido_aplicativo", "lixeira"}, {"id_ped", "id_pedido", "id_cliente", "nome_cliente", "id_local_retirada", "id_local_cobranca", "vendedor_pedido", "vendedor_pedido_id", "listapreco_produtos", "valor_total_produtos", "desconto_pedido", "desconto_pedido_porc", "peso_total_nota", "peso_total_nota_liq", "frete_pedido", "valor_total_nota", "valor_baseICMS", "valor_ICMS", "valor_baseST", "valor_ST", "valor_IPI", "condicao_pagamento_id", "condicao_pagamento", "frete_por_pedido", "transportadora_pedido", "id_transportadora", "data_pedido", "prazo_entrega", "referencia_pedido", "obs_pedido", "obs_interno_pedido", "status_pedido", "contas_pedido", "comissao_pedido", "estoque_pedido", "ordemc_emitido", "data_cad_pedido", "data_mod_pedido", "id_aplicativo", "id_pedido_aplicativo", "lixeira"}),
#"Tipo Alterado" = Table.TransformColumnTypes(data,{{"id_ped", type text}, {"id_pedido", Int64.Type}, {"nome_cliente", type text}, {"valor_total_produtos", type text}}),
#"Valor Substituído" = Table.ReplaceValue(#"Tipo Alterado",".",",",Replacer.ReplaceText,{"valor_total_produtos"}),
#"Tipo Alterado1" = Table.TransformColumnTypes(#"Valor Substituído",{{"valor_total_produtos", Currency.Type}}),
#"Valor Substituído1" = Table.ReplaceValue(#"Tipo Alterado1",".",",",Replacer.ReplaceValue,{"desconto_pedido", "desconto_pedido_porc", "peso_total_nota", "peso_total_nota_liq", "frete_pedido", "valor_total_nota", "valor_baseICMS", "valor_ICMS", "valor_baseST", "valor_ST", "valor_IPI"}),
#"Tipo Alterado2" = Table.TransformColumnTypes(#"Valor Substituído1",{{"desconto_pedido", Currency.Type}, {"desconto_pedido_porc", Currency.Type}, {"peso_total_nota", Currency.Type}, {"peso_total_nota_liq", Currency.Type}, {"frete_pedido", Currency.Type}, {"valor_total_nota", type text}, {"valor_baseICMS", Currency.Type}, {"valor_ICMS", Currency.Type}, {"valor_baseST", Currency.Type}, {"valor_ST", Currency.Type}, {"valor_IPI", Currency.Type}, {"prazo_entrega", type text}, {"data_pedido", type date}}),
#"Colunas Removidas" = Table.RemoveColumns(#"Tipo Alterado2",{"id_aplicativo", "id_pedido_aplicativo", "lixeira"}),
#"Tipo Alterado3" = Table.TransformColumnTypes(#"Colunas Removidas",{{"valor_total_nota", type text}}),
#"Valor Substituído2" = Table.ReplaceValue(#"Tipo Alterado3",".",",",Replacer.ReplaceText,{"valor_total_nota"}),
#"Tipo Alterado4" = Table.TransformColumnTypes(#"Valor Substituído2",{{"valor_total_nota", Currency.Type}})
in
#"Tipo Alterado4"
If I understood your problem correctly of web scraping then I would suggest you don't need to apply for/while loop.
Please, review the below link which might help your issue.
https://www.youtube.com/watch?v=n0bSddoCmss
https://www.myonlinetraininghub.com/scrape-data-multiple-web-pages-power-query
Hey I need help with my code, I am trying to figure out how to deal with different files having different number of pages.
I am actually using power bi to extract data from a folder of pdf's but the deal is that those files have different number of pages, in the code below you can see that I have 3 tables, each of them is on a page(Table1 for the table of the first page..), so for pdf files having only one page the tables 2 and 3 do not exist, so I am looking for a way to ignore parts of code where there will be errors
let
Source = Pdf.Tables(#"Paramètre de l'exemple de fichier1"),
Table1 = Source{[Id="Table1"]}[Data],
#"En-têtes promus1" = Table.PromoteHeaders(Table1, [PromoteAllScalars=true]),
Table2=Source{[Id="Table2"]}[Data],
#"En-têtes promus2" = Table.PromoteHeaders(Table2, [PromoteAllScalars=true]),
Table3=Source{[Id="Table3"]}[Data],
#"En-têtes promus3" = Table.PromoteHeaders(Table3, [PromoteAllScalars=true]),
#"Table" = Table.Combine({#"En-têtes promus1", #"En-têtes promus2",#"En-têtes promus3"}),
#"Index ajouté" = Table.AddIndexColumn(Table, "Index", 1, 1),
#"Colonnes renommées" = Table.RenameColumns(#"Index ajouté",{{"Index", "Page"}})
in
#"Colonnes renommées"
let
Source = Pdf.Tables(#"Paramètre de l'exemple de fichier1"),
Table005 = Source{[Id="Table005"]}[Data],
#"En-têtes promus1" = Table.PromoteHeaders(Table005, [PromoteAllScalars=true]),
#"Table010"=try Source{[Id="Table010"]}[Data] otherwise null,
#"En-têtes promus2" = Table.PromoteHeaders(Table010, [PromoteAllScalars=true]),
Table015=try Source{[Id="Table015"]}[Data] otherwise null ,
#"En-têtes promus3" = Table.PromoteHeaders(Table015, [PromoteAllScalars=true]),
#"Table" = try Table.Combine({#"En-têtes promus1", #"En-têtes promus2"}) otherwise #"En-têtes promus1",
#"Table2" = try Table.Combine({#"Table",#"En-têtes promus3"}) otherwise #"Table",
#"Index ajouté" = Table.AddIndexColumn(Table2, "Index", 1, 1),
#"Colonnes renommées" = Table.RenameColumns(#"Index ajouté",{{"Index", "Page"}})
in
#"Colonnes renommées"
Yes. You can use try and otherwise for steps where errors may occur.
More details:
https://learn.microsoft.com/en-us/power-query/handlingerrors
I have data in the following format, sample shown below:
ValA=101
ValB=2938
ValA=998
ValB=387
ValA=876
ValB=9832
I know that each set of ValA & ValB are a set of values that belong together, so output will be:
ValA ValB
101 2938
998 387
.......
.......
I need to get this into a tabular format so each valA ValB pair is one row.
Ive tried doing this in powerquery by splitting on the = sign and then pivoting on the Val name, but it doesnt work.
any idea on how this might be easily achieved in powerquery?
Thanks!
I ended up doing the exact same as Lukasz, here's the full code:
let
Source = "ValA=101
ValB=2938
ValA=998
ValB=387
ValA=876
ValB=9832",
Custom1 = Lines.FromText(Source),
#"Converted to Table" = Table.FromList(Custom1, Splitter.SplitTextByDelimiter("="), null, null, ExtraValues.Error),
ChangedType = Table.TransformColumnTypes(#"Converted to Table",{{"Column1", type text}, {"Column2", Int64.Type}}),
CustomA = Table.AddColumn(ChangedType, "ValA", each if [Column1] = "ValA" then [Column2] else null),
CustomB = Table.AddColumn(CustomA, "ValB", each if [Column1] = "ValB" then [Column2] else null),
FilledDown = Table.FillDown(CustomB,{"ValA"}),
FilteredRows = Table.SelectRows(FilledDown, each [ValB] <> null)
in
FilteredRows
Lukasz's second idea using pivot columns looks like this:
let
Source = "ValA=101
ValB=2938
ValA=998
ValB=387
ValA=876
ValB=9832",
Custom1 = Lines.FromText(Source),
#"Converted to Table" = Table.FromList(Custom1, Splitter.SplitTextByDelimiter("="), null, null, ExtraValues.Error),
ChangedType = Table.TransformColumnTypes(#"Converted to Table",{{"Column1", type text}, {"Column2", Int64.Type}}),
AddedIndex = Table.AddIndexColumn(ChangedType, "Index", 0, 1),
IntegerDividedColumn = Table.TransformColumns(AddedIndex, {{"Index", each Number.IntegerDivide(_, 2), Int64.Type}}),
PivotedColumn = Table.Pivot(IntegerDividedColumn, List.Distinct(IntegerDividedColumn[Column1]), "Column1", "Column2")
in
PivotedColumn
The trick I found was to add divided-by-two index column (that goes 0, 0, 1, 1, 2, 2...) so the pivot knows the first two rows should be related, and the next two, etc.
You can do the following:
1) create two new calculated columns with logic like if column1 contains ValA then Column1 else null. same logic for ValB in second column.
2) use the fill down feature on the left most column. This will produce rows with values for both ValA and ValB in distinct columns
3) use the filter feature to filter out rows that have nulls in your two new columns
That should give you what you want.
Edit: thinking about this more you might also try: split column1 on the equal sign. Then pivot the new column and it should produce two columns with the discrete values. HTH.