I'm working with Dynamics CRM project operations. I am currently putting together a report based on task effort that is being assigned to resources.
The data being pulled contains a 'planned work' column. These are strings with multiple dates and hours included in the payload.
A single cell example is:
[{""End"":""\/Date(1660838400000)\/"",""Hours"":8,""Start"":""\/Date(1660809600000)\/""},{""End"":""\/Date(1660924800000)\/"",""Hours"":9,""Start"":""\/Date(1660892400000)\/""},{""End"":""\/Date(1661184000000)\/"",""Hours"":9,""Start"":""\/Date(1661151600000)\/""}]
What I need to do is, pull the dates and hours for each entry and add it to a new table so they each have their own rows. Example desired output for this cell:
Start Date
End Date
Hours
1660809600000
1660838400000
8
1660892400000
1660924800000
9
1661151600000
1660924800000
9
The cells can vary in length with multiple entries so it needs to take that into account.
Is there anyone how can point me in the right direction on how this can be done in Power BI?
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45Wiq6OUYpRcs1LAVFWIELfJbEkVcPQzMzAwtjCxAAENPVBEjogwiO/tKgYrNYCzA8uSSwqwabXwNIMSW+tDh57LI1MLHDbY4nfHqBmIu0xNIR6hwx7DA1NDVH8E6sUGwsA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t]),
#"Replaced Value" = Table.ReplaceValue(Source,"""""","""",Replacer.ReplaceText,{"Column1"}),
#"Parsed JSON" = Table.TransformColumns(#"Replaced Value",{},Json.Document),
#"Expanded Column1" = Table.ExpandListColumn(#"Parsed JSON", "Column1"),
#"Expanded Column2" = Table.ExpandRecordColumn(#"Expanded Column1", "Column1", {"End", "Hours", "Start"}, {"End", "Hours", "Start"}),
#"Replaced Value1" = Table.ReplaceValue(#"Expanded Column2","/Date(","",Replacer.ReplaceText,{"End", "Start"}),
#"Replaced Value2" = Table.ReplaceValue(#"Replaced Value1",")/","",Replacer.ReplaceText,{"End", "Start"})
in
#"Replaced Value2"
Adding to #David's answer above, you can also do the two replace value transforms in a single step:
#"Replaced Value1" = Table.ReplaceValue(
#"Expanded Column2",
"blah",
"blah",
(column_value, new, old) => Text.Select(column_value, {"0".."9"}),
{"End", "Start"}
)
Related
Given this sample raw table (there are some more columns..):
agg_group
count_%
CHARGED_OFF
1.2
DELINQUENT
1.8
ELIGIBLE
90
MERCHANT_DELINQ
7
NOT_VERIFIED
0
How can I transform this table, to create 2 new columns, using either DAX in Power BI Desktop, or in Power Query?
Desired result:
agg_group
outstanding_principal
ELIGIBLE
90
DELINQUENT
10
Here is a Power Query sample that does this, paste the following into a blank query to see the steps, based on your sample data:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("JcsxDoAgDIXhu3QmRl3UUaVAEyyRoAsh3P8WNnX93v9qhTPs2aPtyTkwMA0zNFPBYiS+H+SiuCqKeToiCm2jyoVZ/lz638uwqHMq/cVMjtAKStw+", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [agg_group = _t, #"count_%" = _t]),
#"Cleaned Text" = Table.TransformColumns(Source,{{"count_%", each Number.FromText(_, "en-US"), type text}}),
#"Added Conditional Column" = Table.AddColumn(#"Cleaned Text", "agg_group_", each if [#"agg_group"] = "ELIGIBLE" then "ELIGIBLE" else "DELINQUENT"),
#"Grouped Rows" = Table.Group(#"Added Conditional Column", {"agg_group_"}, {{"outstanding_principal", each List.Sum([#"count_%"]), type text}}),
#"Renamed Columns" = Table.RenameColumns(#"Grouped Rows",{{"agg_group_", "agg_group"}})
in
#"Renamed Columns"
Result:
I like to add a column in MainQuery and fill it up with values found in another LookupQuery based what it being able to find the matching unique value embedded in the text of column1 in my Main Query. if not found just return blank, null. Thank you
Main Query
Column1
...111.
..ABC..
..34C..
...xyz.
yyy.....
Lookup Query
Uniques
34C
ABC
111
Desired Output in Main Query
Column1
New Column
...111..
111
..ABC...
ABC
..34C...
34C
....xyz.
yyy.....
Here's another method using List.Accumulate to loop through the lookup list. Not sure which of the methods will be more efficient.
Note: If there might be more than one lookup result for a given item in column 1, a small change in the List.Accumulate function can accommodate showing them all with a separator
let
//Read in Main Query
Source = Excel.CurrentWorkbook(){[Name="Main"]}[Content],
Main = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
//Read in Lookup Query as a list of text items
Lookup = List.Buffer(
List.Transform(Excel.CurrentWorkbook(){[Name="Lookup"]}[Content][Uniques],
each Text.From(_))),
//add Column
#"Add Column" =
Table.AddColumn(
Main, "New Column", each
List.Accumulate(Lookup,
null,
(state, current)=>
if Text.Contains([Column1], current)
then current else state),
type nullable text)
in
#"Add Column"
Results
There's a not a built-in join that does this, but you can cross-join followed by a filter. And to Cross Join in PQ you add a custom column to one table containing the full value of the other table, then run Table.ExpandTableColumn.
EG
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WSjE0NFSK1YlWKk5Jc3RyNrY0BvMSs4uzgDLZxYlZKWnZxibOSrGxAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each Lookup),
#"Expanded Custom" = Table.ExpandTableColumn(#"Added Custom", "Custom", {"Uniques"}, {"Uniques"}),
#"Selected Rows" = Table.SelectRows(#"Expanded Custom", each Text.Contains([Column1], [Uniques]))
in
#"Selected Rows"
How to make a Power Query join of two tables on least difference between columns. I mean absolute difference between numbers.
I followed this great article: https://exceed.hr/blog/merging-with-date-range-using-power-query/
I tried adding this custom column analogously, where L stands for Tab1 and R for Tab2:
= Table.AddColumn(
Source,
"LeastAbsDifference",
(L) =>
Table.SelectRows( Tab2,
(R) => L[category] = R[category] and Number.Abs(L[target] - R[actual]) )
)
It produces error:
Expression.Error: We cannot convert the value 4 to type Logical.
Tables to recreate example:
// Tab1
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTJVitWJVnKCs5whrFgA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [category = _t, target = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"target", Int64.Type}})
in
#"Changed Type"
// Tab2
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTJUitWBsIzgLGMwywnIMoGzTOEsM6XYWAA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [category = _t, actual = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"actual", Int64.Type}})
in
#"Changed Type"
Here's one way:
Append the two tables
Group by Category
Output the desired columns as a Group aggregation
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTJVitWJVnKCs5whrFgA", BinaryEncoding.Base64),
Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [category = _t, target = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"target", Int64.Type}}),
Source2 = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTJUitWBsIzgLGMwywnIMoGzTOEsM6XYWAA=",
BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [category = _t, actual = _t]),
#"Changed Type1" = Table.TransformColumnTypes(Source2,{{"actual", Int64.Type}}),
//append the tables
append = Table.Combine({#"Changed Type",#"Changed Type1"}),
//Group by category, then output the desired columns
#"Grouped Rows" = Table.Group(append, {"category"}, {
{"target", each [target]{0},Int64.Type},
{"actual", (t)=> t[actual]{
List.PositionOf(List.Transform(t[actual], each Number.Abs(t[target]{0} - _)),
List.Min(List.Transform(t[actual], each Number.Abs(t[target]{0} - _))),Occurrence.First)},Int64.Type},
{"least difference", (t)=> List.Min(List.Transform(t[actual], each Number.Abs(t[target]{0} - _))),Int64.Type
}})
in
#"Grouped Rows"
Output from above code
I would like to acknowledge a favor of Kristian Rados, who provided the answer to my question in the comments of his article: Merging with date range using Power Query With my gratitude, and by courtesy of the author, I am quoting the answer in full:
The reason your formula produces an error is the second argument of the Table.SelectRows function. In it, you need to filter the table with the boolean (true/false) expression. In your case, the part of the code with Number.Abs function returns a number instead of true/false (e.g. L[target] – R[actual] = 5-1=4 ). Trying to filter the table this way could be possible, but it would require you to use multiple nested environments, which would result in a very complicated formula and slow performance.
I would suggest trying a different approach. By using your example from stack overflow I reproduced the problem. Below is a complete M code I came up with along with the explanation below:
let
Source = Tab1,
#"Merged Queries" = Table.NestedJoin(Source, {"category"}, Tab2, {"category"}, "Tab2", JoinKind.LeftOuter),
#"Expanded Tab2" = Table.ExpandTableColumn(#"Merged Queries", "Tab2", {"actual"}, {"actual"}),
#"Inserted Subtraction" = Table.AddColumn(#"Expanded Tab2", "Least difference", each Number.Abs([target] - [actual]), Int64.Type),
#"Grouped Rows" = Table.Group(#"Inserted Subtraction", {"category"}, {{"All", each Table.First(Table.Sort(_, {{"Least difference", Order.Ascending}}))}}),
#"Expanded All" = Table.ExpandRecordColumn(#"Grouped Rows", "All", {"target", "actual", "Least difference"}, {"target", "actual", "Least difference"})
in
#"Expanded All"
First, we merged the queries by using the category column. After expanding the table, we subtracted two columns to get the absolute difference between target and actual. Finally, we group by category and sort the table by the Least difference column in ascending order (Table.Sort function inside the grouped rows). After this, we take the first row of the nested table (Table.First function), and finally expand the record column.
I do Power BI for a logistics company. We want to show performance by stop location. The data is currently a table of all orders by Order ID, so -- ID, Rev $, Pickup Stop, Delivery Stop. Everything is a 2-stop load, fortunately.
What I am struggling with is building a calculated table that looks at the Pickup Stop AND the Delivery Stop at the same time while ALSO respecting filters set on the page. I would like the stops table to say something like: Stop Location, X Pickups, $X Pickup Revenue, X Deliveries, $X Delivery Revenue.
How would I go about this? I've tried a number of approaches but every time it either misses filters or can only handle one stop at a time.
Thanks!
Current Datacall it Orders
The calculated table I'm trying to makecall it Stops
One method of creating your Stops, given your Orders is by using Power Query, accessed via Queries=>Transform Data on the Power BI Home Tab.
The Table.Group function is where the magic happens. Unfortunately, it needs to be done by coding in the Advanced Editor, as the UI does not provide for these custom aggregations.
When the PQ Editor opens: Home => Advanced Editor
The first three lines should be replaced by whatever you are reading in your own Orders table with.
Paste the rest of M Code below in place of what is below your setup lines in your own query
Read the comments and explore the Applied Steps to understand the algorithm
M Code
let
//Input data and set datatypes
//These lines should be replaced with whatever you need to
//set up your data table
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("bYzBCoMwEER/Zck5BxPRu1RaLJaW6qEQPIS4tEExoonQv+/a0oLQyw5vZnaUYuepxQmKnHFWO697uOKCQ0DiizVdGKHybiTKsbcLTs8PN1wxIZMooiR938z3evCawyFbKczeDhzq268qyBZpsg23f9+qJF+Skuwe1ui741CU/2djsmO53lJ3SFsth/3aPWrTzY7Kp4o1zQs=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t, Column2 = _t, Column3 = _t, Column4 = _t]),
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
dataSource = Table.TransformColumnTypes(#"Promoted Headers",{
{"Order ID", Int64.Type}, {"Total Revenue", Int64.Type},
{"Pickup Stop", type text}, {"Delivery Stop", type text}}),
//Unpivot to get single column of Stops
#"Unpivoted Columns" = Table.UnpivotOtherColumns(dataSource, {"Order ID", "Total Revenue"}, "Attribute", "Stop"),
//Group by stop and do the aggregations
#"Grouped Rows" = Table.Group(#"Unpivoted Columns", {"Stop"}, {
{"Orders Picked Up", (t)=> List.Count(List.Select(t[Attribute], each _ = "Pickup Stop" )), Int64.Type},
{"Total Revenue Picked Up", (t)=> List.Sum(Table.SelectRows(t, each [Attribute]="Pickup Stop")[Total Revenue]), type number},
{"Orders Delivered", (t)=> List.Count(List.Select(t[Attribute], each _ = "Delivery Stop" )), Int64.Type},
{"Total Revenue Delivered", (t)=> List.Sum(Table.SelectRows(t, each [Attribute]="Delivery Stop")[Total Revenue]), type number}
})
in
#"Grouped Rows"
Orders
Stops
I have the below chart
that is sorted by the count for each of the bins. I try to sort it by days open starting from 0 - 5 days, 5 - 10 days etc...
I have added another table that has IDs for each of the bins (0 - 5 days is 1, 5 - 10 days is 2) but I am unable to use it to sort it.
Any ideas?
I do it always by adding dimension table for sorting purpose. Dim table for bins would look like this:
Then go to Data pane and set it up as shown in the picture below.
Select Bin name column
Choose Modeling from menu
Sort by column and here choose column Bin order
Then connect the Dim table to fact table:
While making visual choose Bin name from Dim Table not Fact Table!
Then the final thing is set up sorting in visual:
Here you have Dim and Fact table to reproduce exercise.
Dim Table:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WcsrMUzDQNVXSUTJUitWB8E11DQ2AAkZwAUMDXSOQiDFcxAioByRigtBkoGsIVmSK0GZkoA0UMFOKjQUA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [#"Bin name" = _t, #"Bin order" = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Bin name", type text}, {"Bin order", Int64.Type}})
in
#"Changed Type"
Fact Table:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("jdAxDoAgDAXQq5iu0qQtVmX1GoQDuHj/0SoJCWVh5KefR8kZrvtZCBUCMJRQz4pMFkgLmFC+ZGuJWKefUUL+h6zbekJrd3OV1EvHhBSnJHLU6ak0QaWRil5SBw2/pwPEMzvtHrLnlRc=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [#"Bin name" = _t, Frequency = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Bin name", type text}, {"Frequency", Int64.Type}})
in
#"Changed Type"
You should be able to do a Sort by Column under the Modeling tab where you sort your bin name column by the ID value column.
You need to :
Connect the bins (0-5),(5-10) columns from the two tables in your relationships.
In your second table, add a column called order: 1,2,3 for the bins (0-5), 5-10 respectively and so on
This should work