I have data as such:
I want to split the data in the following format based on date. So going forward the dates can be split and I can use slicer to select the range and get the index value over each selected dates.
Please help on this.
Your pivoted result does not make sense. The first row for example, combines rows #1 and #8 from your table, but there is a prefix 17/22/38 in row #8 which makes the URL different than the one from row #1. Where did this prefix go? How it disappeared and why? And this is also for the other rows, e.g. Contact-us.
But otherwise, Pivot columns is what you need. If your original table looks like this:
Select Date column and click Pivot column command in the ribbon, and you will get a dialog like this:
Select Index as values column. In you case probably it makes no sense to aggregate indexes, so select Don't Aggregate. This will give you a result like this:
Which is as close to your desired result, as possible (considering the issue that I mentioned above).
And here is the M code to reproduce the steps above:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("hdE9CoAwDIDRq0jmUpL0z55FOujsJAge36CQycSl3/IgJF0WoB6xR0bqEGDdLnnPY59ISjCCCViaPJCk7IEszR4o0uKBKm0eaNL6AMZPMOsIA3Rd0wCEfzOIdFFLsN7KEkn/wxL5Pca4AQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [Date = _t, Search_Term = _t, Url = _t, Index = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Search_Term", type text}, {"Url", type text}, {"Index", Int64.Type}}),
#"Pivoted Column" = Table.Pivot(Table.TransformColumnTypes(#"Changed Type", {{"Date", type text}}, "en-US"), List.Distinct(Table.TransformColumnTypes(#"Changed Type", {{"Date", type text}}, "en-US")[Date]), "Date", "Index")
in
#"Pivoted Column"
Related
I like to add a column in MainQuery and fill it up with values found in another LookupQuery based what it being able to find the matching unique value embedded in the text of column1 in my Main Query. if not found just return blank, null. Thank you
Main Query
Column1
...111.
..ABC..
..34C..
...xyz.
yyy.....
Lookup Query
Uniques
34C
ABC
111
Desired Output in Main Query
Column1
New Column
...111..
111
..ABC...
ABC
..34C...
34C
....xyz.
yyy.....
Here's another method using List.Accumulate to loop through the lookup list. Not sure which of the methods will be more efficient.
Note: If there might be more than one lookup result for a given item in column 1, a small change in the List.Accumulate function can accommodate showing them all with a separator
let
//Read in Main Query
Source = Excel.CurrentWorkbook(){[Name="Main"]}[Content],
Main = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
//Read in Lookup Query as a list of text items
Lookup = List.Buffer(
List.Transform(Excel.CurrentWorkbook(){[Name="Lookup"]}[Content][Uniques],
each Text.From(_))),
//add Column
#"Add Column" =
Table.AddColumn(
Main, "New Column", each
List.Accumulate(Lookup,
null,
(state, current)=>
if Text.Contains([Column1], current)
then current else state),
type nullable text)
in
#"Add Column"
Results
There's a not a built-in join that does this, but you can cross-join followed by a filter. And to Cross Join in PQ you add a custom column to one table containing the full value of the other table, then run Table.ExpandTableColumn.
EG
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WSjE0NFSK1YlWKk5Jc3RyNrY0BvMSs4uzgDLZxYlZKWnZxibOSrGxAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each Lookup),
#"Expanded Custom" = Table.ExpandTableColumn(#"Added Custom", "Custom", {"Uniques"}, {"Uniques"}),
#"Selected Rows" = Table.SelectRows(#"Expanded Custom", each Text.Contains([Column1], [Uniques]))
in
#"Selected Rows"
How to make a Power Query join of two tables on least difference between columns. I mean absolute difference between numbers.
I followed this great article: https://exceed.hr/blog/merging-with-date-range-using-power-query/
I tried adding this custom column analogously, where L stands for Tab1 and R for Tab2:
= Table.AddColumn(
Source,
"LeastAbsDifference",
(L) =>
Table.SelectRows( Tab2,
(R) => L[category] = R[category] and Number.Abs(L[target] - R[actual]) )
)
It produces error:
Expression.Error: We cannot convert the value 4 to type Logical.
Tables to recreate example:
// Tab1
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTJVitWJVnKCs5whrFgA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [category = _t, target = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"target", Int64.Type}})
in
#"Changed Type"
// Tab2
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTJUitWBsIzgLGMwywnIMoGzTOEsM6XYWAA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [category = _t, actual = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"actual", Int64.Type}})
in
#"Changed Type"
Here's one way:
Append the two tables
Group by Category
Output the desired columns as a Group aggregation
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTJVitWJVnKCs5whrFgA", BinaryEncoding.Base64),
Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [category = _t, target = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"target", Int64.Type}}),
Source2 = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTJUitWBsIzgLGMwywnIMoGzTOEsM6XYWAA=",
BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [category = _t, actual = _t]),
#"Changed Type1" = Table.TransformColumnTypes(Source2,{{"actual", Int64.Type}}),
//append the tables
append = Table.Combine({#"Changed Type",#"Changed Type1"}),
//Group by category, then output the desired columns
#"Grouped Rows" = Table.Group(append, {"category"}, {
{"target", each [target]{0},Int64.Type},
{"actual", (t)=> t[actual]{
List.PositionOf(List.Transform(t[actual], each Number.Abs(t[target]{0} - _)),
List.Min(List.Transform(t[actual], each Number.Abs(t[target]{0} - _))),Occurrence.First)},Int64.Type},
{"least difference", (t)=> List.Min(List.Transform(t[actual], each Number.Abs(t[target]{0} - _))),Int64.Type
}})
in
#"Grouped Rows"
Output from above code
I would like to acknowledge a favor of Kristian Rados, who provided the answer to my question in the comments of his article: Merging with date range using Power Query With my gratitude, and by courtesy of the author, I am quoting the answer in full:
The reason your formula produces an error is the second argument of the Table.SelectRows function. In it, you need to filter the table with the boolean (true/false) expression. In your case, the part of the code with Number.Abs function returns a number instead of true/false (e.g. L[target] – R[actual] = 5-1=4 ). Trying to filter the table this way could be possible, but it would require you to use multiple nested environments, which would result in a very complicated formula and slow performance.
I would suggest trying a different approach. By using your example from stack overflow I reproduced the problem. Below is a complete M code I came up with along with the explanation below:
let
Source = Tab1,
#"Merged Queries" = Table.NestedJoin(Source, {"category"}, Tab2, {"category"}, "Tab2", JoinKind.LeftOuter),
#"Expanded Tab2" = Table.ExpandTableColumn(#"Merged Queries", "Tab2", {"actual"}, {"actual"}),
#"Inserted Subtraction" = Table.AddColumn(#"Expanded Tab2", "Least difference", each Number.Abs([target] - [actual]), Int64.Type),
#"Grouped Rows" = Table.Group(#"Inserted Subtraction", {"category"}, {{"All", each Table.First(Table.Sort(_, {{"Least difference", Order.Ascending}}))}}),
#"Expanded All" = Table.ExpandRecordColumn(#"Grouped Rows", "All", {"target", "actual", "Least difference"}, {"target", "actual", "Least difference"})
in
#"Expanded All"
First, we merged the queries by using the category column. After expanding the table, we subtracted two columns to get the absolute difference between target and actual. Finally, we group by category and sort the table by the Least difference column in ascending order (Table.Sort function inside the grouped rows). After this, we take the first row of the nested table (Table.First function), and finally expand the record column.
I wrote a query that pulls data into Power BI. I was wondering if I can create another query that pulls in the original data without certain columns. I know that I can delete a column but I was wondering if I can remove a column and have other columns aggregated. I want to do this from the back-end (PowerQuery). I know I can create another query without including the other column but since this is real-time data, I need to pull the data from the original query.
This is the original data.
This is what I am trying to achieve. I want to remove the column 'Code' but as well as having the other columns aggregated (Calls, Invalid) and distinct columns (Date, Name, Connection Type).
Is this possible on the power query?!
Of course it is possible. Here is an example M code how to do that in Power Query:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("ndKxCoMwEAbgd8ksohdN5y527VDoIA5BQwnYCufSx28oKZHmzqQdJEL4uPvv0vcCoDyUUEElCnExT726s3bf1aKZ3Hm8G7Sjdn/yfTMUtAHSNKQ5mQVvVrOV2oT6r5b0ajbrmlHurNF+B+tQP0bj++aJzCfdvKCdtqGi9iAB2Vz0wgJsyDFu+qz5pxEV22dsuH0myQ4lacKi2x9yBaVIBaSKt5bTYbwy9tmT6pMrGqJKQMVBP5PhBQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Date = _t, Name = _t, Code = _t, #"Connection Type" = _t, Country = _t, Calls = _t, Invalid = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Name", type text}, {"Code", type text}, {"Connection Type", type text}, {"Country", type text}, {"Calls", Int64.Type}, {"Invalid", Int64.Type}}),
#"Removed Columns" = Table.RemoveColumns(#"Changed Type",{"Code"}),
#"Grouped Rows" = Table.Group(#"Removed Columns", {"Date", "Name", "Connection Type", "Country"}, {{"Calls", each List.Sum([Calls]), type text}, {"Invalid", each List.Sum([Invalid]), type text}})
in
#"Grouped Rows"
Table.RemoveColumns will remove the Code column and Table.Group will group the data on the specified columns (Date, Name, Connection Type and Country) and aggregate the data, sum in this case (Calls and Invalid).
You can do this using the UI only. In Power Query Editor, right click the title of Code column and select Remove. Then from Transform tab click on the leftmost button Group By and fill it as follows:
I'm working with the dataset where the gender of all participants is described in one single column. I'd like to create two new columns for both genders and fill it up with the number of males / females involved.
The syntax used in a column looks like this:
0::Male||1::Male||3::Male||4::Female
(so we have 4 participants, the value in col "Male" would be 3, in col "Female" 1)
Would you be so kind and help me to extract this information? ♥
I'm sorry to ask you as I know I'd be able to eventually find the solution by myself, but I'm really under pressure right now. :/
This is the screenshot of the column I wanna extract values from.
Thanks a lot to everyone who tries to help! :)
In powerquery, you'd need two splits (one on :: and one on ||, and then a pivot to get data into right format)
Select your data, including header column of participant_gender and load your table into powerquery with Data ... From Table/Range...[x] my data has headers. I assume below that this is the first table of your file, and is named Table1
Add Column..Index Column...
right click participant_gender column ... split column ... by delimiter ... custom ... || [x] each occurrence of the delimiter , Advanced options [x] rows
right click participant_gender column ... split column ... by delimiter ... custom ... :: [x] each occurrence of the delimiter (ignore advanced options)
Click to select both participant_gender.2 and Index columns .. right click group by ... group by (participant_gender.2) (index) new column name (count) operation (sum) column (participant_gender.1)
Click to select participant_Gender.2 column ... Transform..pivot column...Values column (Count)
File...Close and Load to ... table
Code produced, which you can paste into Home .. Advanced Editor... if you wish
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"participant_gender", type text}}),
#"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 0, 1),
#"Split Column by Delimiter" = Table.ExpandListColumn(Table.TransformColumns(#"Added Index", {{"participant_gender", Splitter.SplitTextByDelimiter("||", QuoteStyle.Csv), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "participant_gender"),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"participant_gender", type text}}),
#"Split Column by Delimiter1" = Table.SplitColumn(#"Changed Type1", "participant_gender", Splitter.SplitTextByDelimiter("::", QuoteStyle.Csv), {"participant_gender.1", "participant_gender.2"}),
#"Changed Type2" = Table.TransformColumnTypes(#"Split Column by Delimiter1",{{"participant_gender.1", Int64.Type}, {"participant_gender.2", type text}}),
#"Grouped Rows" = Table.Group(#"Changed Type2", {"participant_gender.2", "Index"}, {{"Count", each List.Sum([participant_gender.1]), type number}}),
#"Pivoted Column" = Table.Pivot(#"Grouped Rows", List.Distinct(#"Grouped Rows"[participant_gender.2]), "participant_gender.2", "Count", List.Sum)
in #"Pivoted Column"
In Power BI, I have a table that looks like this:
ID
234
435
3435
58
48504
7820
I want to convert it to a table that looks like this:
ID
234-101
234-102
435-101
435-102
3435-101
343-102
58-101
58-102
48504-101
48504-102
7820-101
7820-102
Is this even possible within Power BI?
I thought of two ways to do this, though there are probably others.
NOTE - I prefer the second method as it lets the "101" and "102" be data driven allowing them to be changed or added to in the future more easily.
A) Through the Query Editor (requires hard-coding the "101"/"102" values)
Step 1: Start with your data in the Query Editor
Step 2: Add two additional columns for your suffixes. Click on the "Custom Column from Examples" button and then type in "234-101" in the first cell. After arrowing down to the next cell, it should auto-populate the rest. Do this again for "-102".
Step 3: Unpivot the two new columns to get them into one. With the "ID" column selected, click on the dropdown for "Unpivot Columns" and click on "Unpivot Other Columns".
Step 4: Remove extra columns. In the resulting data, you will have the original "ID" column, along with two new ones; "Attribute" and "Value". Since the "Value" column contains the desired values, select the "ID" and "Attribute" columns, right click one of their headers, and select "Remove Columns".
Step 5: Rename the "Value" column to "ID" and you're finished.
Here is the resulting M code for all of those actions.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMjI2UYrViVYyMTYF08YwhqkFRNzC1ACiwtzCyEApNhYA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [ID = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}}),
#"Inserted Merged Column" = Table.AddColumn(#"Changed Type", "Merged", each Text.Combine({Text.From([ID], "en-US"), "-101"}), type text),
#"Inserted Merged Column1" = Table.AddColumn(#"Inserted Merged Column", "Merged.1", each Text.Combine({Text.From([ID], "en-US"), "-102"}), type text),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Inserted Merged Column1", {"ID"}, "Attribute", "Value"),
#"Removed Columns" = Table.RemoveColumns(#"Unpivoted Other Columns",{"ID", "Attribute"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Value", "ID"}})
in
#"Renamed Columns"
B) Through DAX
Step 1: Start with your data in the data view.
Step 2: Click on "Enter Data" and add the data for the suffixes. (Skip this if these numbers are sourced somewhere else)
Step 3: Click on "New Table" and enter the following formula.
NewData = CROSSJOIN(Data, Suffixes)
Step 4: Click on "New Column and enter the following formula.
NewID = CONCATENATE(CONCATENATE(NewData[ID], "-"), NewData[Value])
If you want the new column to be named "ID", you'll need to rename the old "ID" column first, as you can't simply remove it like was done in the first method.
If you're okay with using Power BI's query editor (Power Query) for this, you can do it with this query code:
let
Source = Table1,
#"Inserted Merged Column3" = Table.AddColumn(Source, "DelimitedListWithSuffixes", each Text.Combine({[ID], "-101,", [ID],"-102"}), type text),
#"Split Column by Delimiter" = Table.ExpandListColumn(Table.TransformColumns(#"Inserted Merged Column3", {{"DelimitedListWithSuffixes", Splitter.SplitTextByDelimiter(",", QuoteStyle.Csv), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "DelimitedListWithSuffixes"),
#"Changed Type" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"DelimitedListWithSuffixes", type text}}),
#"Removed Other Columns" = Table.SelectColumns(#"Changed Type",{"DelimitedListWithSuffixes"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Other Columns",{{"DelimitedListWithSuffixes", "ID"}})
in
#"Renamed Columns"
(Table1 is your original ID column table.)