Given this sample raw table (there are some more columns..):
agg_group
count_%
CHARGED_OFF
1.2
DELINQUENT
1.8
ELIGIBLE
90
MERCHANT_DELINQ
7
NOT_VERIFIED
0
How can I transform this table, to create 2 new columns, using either DAX in Power BI Desktop, or in Power Query?
Desired result:
agg_group
outstanding_principal
ELIGIBLE
90
DELINQUENT
10
Here is a Power Query sample that does this, paste the following into a blank query to see the steps, based on your sample data:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("JcsxDoAgDIXhu3QmRl3UUaVAEyyRoAsh3P8WNnX93v9qhTPs2aPtyTkwMA0zNFPBYiS+H+SiuCqKeToiCm2jyoVZ/lz638uwqHMq/cVMjtAKStw+", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [agg_group = _t, #"count_%" = _t]),
#"Cleaned Text" = Table.TransformColumns(Source,{{"count_%", each Number.FromText(_, "en-US"), type text}}),
#"Added Conditional Column" = Table.AddColumn(#"Cleaned Text", "agg_group_", each if [#"agg_group"] = "ELIGIBLE" then "ELIGIBLE" else "DELINQUENT"),
#"Grouped Rows" = Table.Group(#"Added Conditional Column", {"agg_group_"}, {{"outstanding_principal", each List.Sum([#"count_%"]), type text}}),
#"Renamed Columns" = Table.RenameColumns(#"Grouped Rows",{{"agg_group_", "agg_group"}})
in
#"Renamed Columns"
Result:
Related
I have rows of data which can have information in multiple columns that I need to extract and convert into an individual row for each.
E.g.
Original table
Headers are:
Product Code | Description | Location 1 | Location 2 | Location 3
and I need to convert it to:
Product Code | Description | Location
Some products will be available in multiple regions.
If a product is available in Germany and France, there may be an DE in the Location 1 column, and an FR in the Location 2 column, while the location 3 column will be blank.
I need to convert it so that there is a single location column with corresponding entries for each region that product had.
Desired output table
Is there a way to automate this in Power Bi?
Select the Code and description columns then
UnpivotOtherColumns
Remove the blank entries
Remove the Attribute column
Not sure how you want your results sorted, but you could easily add a sorting algorithm to below.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUfJILAGSLq5AAoRidaKVjICM4OTEojQg7RYElwVJGQMZ7jn5ZanFQEaoN5KC2FgA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Product Code" = _t, Description = _t, #"Location 1" = _t, #"Location 2" = _t, #"Location 3" = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Product Code", Int64.Type}, {"Description", type text}, {"Location 1", type text}, {"Location 2", type text}, {"Location 3", type text}}),
//Select Product Code and Description Columns
// Then "Unpivot other Columns
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Changed Type",
{"Product Code", "Description"}, "Attribute", "Location"),
//Remove the blank locations and the "Attrubute" column
#"Filtered Rows" = Table.SelectRows(#"Unpivoted Other Columns", each ([Location] <> "")),
#"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"Attribute"})
in
#"Removed Columns"
How to make a Power Query join of two tables on least difference between columns. I mean absolute difference between numbers.
I followed this great article: https://exceed.hr/blog/merging-with-date-range-using-power-query/
I tried adding this custom column analogously, where L stands for Tab1 and R for Tab2:
= Table.AddColumn(
Source,
"LeastAbsDifference",
(L) =>
Table.SelectRows( Tab2,
(R) => L[category] = R[category] and Number.Abs(L[target] - R[actual]) )
)
It produces error:
Expression.Error: We cannot convert the value 4 to type Logical.
Tables to recreate example:
// Tab1
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTJVitWJVnKCs5whrFgA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [category = _t, target = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"target", Int64.Type}})
in
#"Changed Type"
// Tab2
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTJUitWBsIzgLGMwywnIMoGzTOEsM6XYWAA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [category = _t, actual = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"actual", Int64.Type}})
in
#"Changed Type"
Here's one way:
Append the two tables
Group by Category
Output the desired columns as a Group aggregation
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTJVitWJVnKCs5whrFgA", BinaryEncoding.Base64),
Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [category = _t, target = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"target", Int64.Type}}),
Source2 = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTJUitWBsIzgLGMwywnIMoGzTOEsM6XYWAA=",
BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [category = _t, actual = _t]),
#"Changed Type1" = Table.TransformColumnTypes(Source2,{{"actual", Int64.Type}}),
//append the tables
append = Table.Combine({#"Changed Type",#"Changed Type1"}),
//Group by category, then output the desired columns
#"Grouped Rows" = Table.Group(append, {"category"}, {
{"target", each [target]{0},Int64.Type},
{"actual", (t)=> t[actual]{
List.PositionOf(List.Transform(t[actual], each Number.Abs(t[target]{0} - _)),
List.Min(List.Transform(t[actual], each Number.Abs(t[target]{0} - _))),Occurrence.First)},Int64.Type},
{"least difference", (t)=> List.Min(List.Transform(t[actual], each Number.Abs(t[target]{0} - _))),Int64.Type
}})
in
#"Grouped Rows"
Output from above code
I would like to acknowledge a favor of Kristian Rados, who provided the answer to my question in the comments of his article: Merging with date range using Power Query With my gratitude, and by courtesy of the author, I am quoting the answer in full:
The reason your formula produces an error is the second argument of the Table.SelectRows function. In it, you need to filter the table with the boolean (true/false) expression. In your case, the part of the code with Number.Abs function returns a number instead of true/false (e.g. L[target] – R[actual] = 5-1=4 ). Trying to filter the table this way could be possible, but it would require you to use multiple nested environments, which would result in a very complicated formula and slow performance.
I would suggest trying a different approach. By using your example from stack overflow I reproduced the problem. Below is a complete M code I came up with along with the explanation below:
let
Source = Tab1,
#"Merged Queries" = Table.NestedJoin(Source, {"category"}, Tab2, {"category"}, "Tab2", JoinKind.LeftOuter),
#"Expanded Tab2" = Table.ExpandTableColumn(#"Merged Queries", "Tab2", {"actual"}, {"actual"}),
#"Inserted Subtraction" = Table.AddColumn(#"Expanded Tab2", "Least difference", each Number.Abs([target] - [actual]), Int64.Type),
#"Grouped Rows" = Table.Group(#"Inserted Subtraction", {"category"}, {{"All", each Table.First(Table.Sort(_, {{"Least difference", Order.Ascending}}))}}),
#"Expanded All" = Table.ExpandRecordColumn(#"Grouped Rows", "All", {"target", "actual", "Least difference"}, {"target", "actual", "Least difference"})
in
#"Expanded All"
First, we merged the queries by using the category column. After expanding the table, we subtracted two columns to get the absolute difference between target and actual. Finally, we group by category and sort the table by the Least difference column in ascending order (Table.Sort function inside the grouped rows). After this, we take the first row of the nested table (Table.First function), and finally expand the record column.
I am currently using Power BI (Power Query) to clean up a dataset, and I am having some problems unpivotting a table the right way (see image below). Any suggestions on how to sort this out?
For unpivoting to work you need three columns:
Key
Column
Value
1
A
1.3
1
B
3
1
C
New
2
A
2.3
2
B
3
2
C
Old
So this could be unpivoted to:
Key
Column A
Column B
Column C
1
1.3
3
New
2
2.3
3
Old
So for this you would need to add a column-column where the row value is the column name - prior to unpivoting the table.
Edit:
If your rows are ordered according to columns per key, you can do something like this, where you create your own column-column prior to pivoting. See code you can paste into a blank query:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("Tcq7DQAgCAXAXV5tI0zAZwvC/muYaCSvu+KqsLFgZuj17O7jiBhn5rXQF/pCX+grfaWv9PX/Pg==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [id = _t, values = _t]),
#"Added Index" = Table.AddIndexColumn(Source, "Index", 1, 1, Int64.Type),
#"Added Custom" = Table.AddColumn(#"Added Index", "Custom", each Number.Mod([Index],4)),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Index"}),
#"Pivoted Column1" = Table.Pivot(Table.TransformColumnTypes(#"Removed Columns", {{"Custom", type text}}, "nb-NO"), List.Distinct(Table.TransformColumnTypes(#"Removed Columns", {{"Custom", type text}}, "nb-NO")[Custom]), "Custom", "values")
in
#"Pivoted Column1"
I want to move the data from one column which has distinct values in rows but the other column has same values in rows as shown in the pitcure.
I am new to power BI, can someone please help on this?
Thank you in advance
Req
You can create the following two measures - using CONCATENATEX()
Col_1 = CONCATENATEX(Your_Table,Your_Table[Col1]&";")
Col_2 = CONCATENATEX(Your_Table,Your_Table[Col2]&";")
In Power BI, you can perform some Transformation in Power Query Editor to achieve your required output.
Let, your table structure and sample data looks as below-
Now, go to Advanced Editor for your table and replace the code with this below code-
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("PcdJDoAgEETRu/TaSyCoKDjPEu5/Dbs0qcVP/ktJjBTitENy8avSTqrWLqrRbsprz6dSr9WMITuwJANoyQg6sgcrcgDrj1Z3BBtyAj05gy25gB25goHcwEjuYC85vw==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [col1 = _t, col2 = _t, col3 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"col1", type text}, {"col2", type text}, {"col3", type text}}),
//--New Transformation starts from here
#"Grouped Rows" = Table.Group(#"Changed Type", {"col1"}, {{"grouped", each _, type table [col1=nullable text, col2=nullable text, col3=nullable text]}}),
#"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each [grouped][col2]),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom.1", each [grouped][col3]),
#"Extracted Values" = Table.TransformColumns(#"Added Custom1", {"Custom", each Text.Combine(List.Transform(_, Text.From), ";"), type text}),
#"Extracted Values1" = Table.TransformColumns(#"Extracted Values", {"Custom.1", each Text.Combine(List.Transform(_, Text.From), ";"), type text}),
#"Removed Columns" = Table.RemoveColumns(#"Extracted Values1",{"grouped"})
in
#"Removed Columns"
Here is the final output-
I wrote a query that pulls data into Power BI. I was wondering if I can create another query that pulls in the original data without certain columns. I know that I can delete a column but I was wondering if I can remove a column and have other columns aggregated. I want to do this from the back-end (PowerQuery). I know I can create another query without including the other column but since this is real-time data, I need to pull the data from the original query.
This is the original data.
This is what I am trying to achieve. I want to remove the column 'Code' but as well as having the other columns aggregated (Calls, Invalid) and distinct columns (Date, Name, Connection Type).
Is this possible on the power query?!
Of course it is possible. Here is an example M code how to do that in Power Query:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("ndKxCoMwEAbgd8ksohdN5y527VDoIA5BQwnYCufSx28oKZHmzqQdJEL4uPvv0vcCoDyUUEElCnExT726s3bf1aKZ3Hm8G7Sjdn/yfTMUtAHSNKQ5mQVvVrOV2oT6r5b0ajbrmlHurNF+B+tQP0bj++aJzCfdvKCdtqGi9iAB2Vz0wgJsyDFu+qz5pxEV22dsuH0myQ4lacKi2x9yBaVIBaSKt5bTYbwy9tmT6pMrGqJKQMVBP5PhBQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Date = _t, Name = _t, Code = _t, #"Connection Type" = _t, Country = _t, Calls = _t, Invalid = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Name", type text}, {"Code", type text}, {"Connection Type", type text}, {"Country", type text}, {"Calls", Int64.Type}, {"Invalid", Int64.Type}}),
#"Removed Columns" = Table.RemoveColumns(#"Changed Type",{"Code"}),
#"Grouped Rows" = Table.Group(#"Removed Columns", {"Date", "Name", "Connection Type", "Country"}, {{"Calls", each List.Sum([Calls]), type text}, {"Invalid", each List.Sum([Invalid]), type text}})
in
#"Grouped Rows"
Table.RemoveColumns will remove the Code column and Table.Group will group the data on the specified columns (Date, Name, Connection Type and Country) and aggregate the data, sum in this case (Calls and Invalid).
You can do this using the UI only. In Power Query Editor, right click the title of Code column and select Remove. Then from Transform tab click on the leftmost button Group By and fill it as follows: