Above is my current input.
Below is the desired output.
Currently, i load the data into Power Bi, and using the Power Query Editor I 'unpivot columns' on 'selected columns only'. This results in a column of mixed data types (Integers and strings). Is anyone able to advise an efficient method upon data load how to separate string values and integers?
Below is the code from Advanced Editor
let
Source = Csv.Document(File.Contents("F:\Surveys\dev\pivottest.csv"),[Delimiter=",", Columns=5, Encoding=1252, QuoteStyle=QuoteStyle.None]),
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"responseid", Int64.Type}, {"q1", Int64.Type}, {"q2", type text}, {"q3", Int64.Type}, {"q4", type text}}),
#"Unpivoted Only Selected Columns" = Table.Unpivot(#"Changed Type", {"q1", "q2", "q3", "q4"}, "Attribute", "Value")
in
#"Unpivoted Only Selected Columns"
After you have unpivoted your data, then use the function Value.Is on your Value column. To check if a value is number, it would look something like this:
Value.Is([Column], Int64.Type)
Example,
IsNumber =Value.Is(Value.FromText([ColumnOfMixedValues]), type number)
To check for a text, it would be:
Value.Is([Column],type text)
Example,
IsText =Value.Is(Value.FromText([ColumnOfMixedValues]), type text)
Value.Is returns a boolean true/false, so you can wrap it in an IF.
if Value.Is(AlphaNumeric, type text) then AlphaNumeric else Number.ToText(AlphaNumeric))
Related
First of all, thank you for reading this question
So, I have a set of data which is the difference between two date in "minutes" the raw data type is "Whole Number"
I would like to convert it to be "DD:HH:MM", not sure if anyone could help me, thank you so much!
AFAIK, Power BI does not have or recognize a duration type. If you want to display in that format, it will need to be text.
In Power Query you can transform that column to text with code such as:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45W0jU0NDVUitUBskwtTSAMYwhtCSZNzCAcCzBlZAjhGhqbQATMzC3NlWJjAQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Minutes = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Minutes", Int64.Type}}),
#"Transform to Duration" = Table.TransformColumns(#"Changed Type", {"Minutes", each Duration.ToText(#duration(0,0,_,0)), type text})
in
#"Transform to Duration"
Then, in Desktop, format that column as Text
If you need to do calculations on the duration, you may want to add a column instead of transforming the column, or do the calculations in Power Query.
//You can use this
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"duration", Int64.Type}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each [duration]/1440),
#"Changed Type1" = Table.TransformColumnTypes(#"Added Custom",{{"Custom", type duration}})
in
#"Changed Type1"
I am working with data that is structured in a parent-child relationship. Every level already has the rolled-up value (sum of the children). Therefore, I want Power Bi to display the value that is shown at every level (or sum on the same level) and not aggregate between parent and child. Also, the sums of the children don't always equal the parents, because some details are missing at the lower level. I do still need the parent-child relationship in PowerBi for drill through purposes. Attached is an example of how the data is structured. The table on the left is how I receive it. I have a dynamic number of levels, so ideally the solution wouldn't have a hard-coded number of levels. Any thoughts?
There's no way around having a fixed number of levels in PowerBI, but you can easily write a measure that takes the MAX rather than the SUM of the values.
You can apply these below steps to your table in Advanced Editor to get your desired output. Please replace the previous_step_name name (my frst step) accordingly to make this code functional.
let
//Your previous steps,
#"Added Index" = Table.AddIndexColumn(#"previous_step_name", "Index", 1, 1, Int64.Type),
tab_before_split = Table.ReorderColumns(#"Added Index",{"Index", "Item", "Value"}),
tab_split = Table.ExpandListColumn(Table.TransformColumns(#"tab_before_split", {{"Item", Splitter.SplitTextByDelimiter("/", QuoteStyle.Csv), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "Item"),
tab_group_by = Table.Group(#"tab_split",{"Index"}, {{"Count", each Table.RowCount(_), Int64.Type}}),
tab_new = Table.Join(tab_before_split,"Index", tab_group_by,"Index"),
#"Split Column by Delimiter" = Table.SplitColumn(tab_new, "Item", Splitter.SplitTextByDelimiter("/", QuoteStyle.Csv), {"Item.1", "Item.2", "Item.3"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"Item.1", type text}, {"Item.2", type text}, {"Item.3", type text}}),
#"Added Custom" = Table.AddColumn(#"Changed Type1", "Level", each [Count] - 1),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Count"}),
#"Changed Type2" = Table.TransformColumnTypes(#"Removed Columns",{{"Level", Int64.Type}})
in
#"Changed Type2"
Sample input-
Sample output-
I wrote a query that pulls data into Power BI. I was wondering if I can create another query that pulls in the original data without certain columns. I know that I can delete a column but I was wondering if I can remove a column and have other columns aggregated. I want to do this from the back-end (PowerQuery). I know I can create another query without including the other column but since this is real-time data, I need to pull the data from the original query.
This is the original data.
This is what I am trying to achieve. I want to remove the column 'Code' but as well as having the other columns aggregated (Calls, Invalid) and distinct columns (Date, Name, Connection Type).
Is this possible on the power query?!
Of course it is possible. Here is an example M code how to do that in Power Query:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("ndKxCoMwEAbgd8ksohdN5y527VDoIA5BQwnYCufSx28oKZHmzqQdJEL4uPvv0vcCoDyUUEElCnExT726s3bf1aKZ3Hm8G7Sjdn/yfTMUtAHSNKQ5mQVvVrOV2oT6r5b0ajbrmlHurNF+B+tQP0bj++aJzCfdvKCdtqGi9iAB2Vz0wgJsyDFu+qz5pxEV22dsuH0myQ4lacKi2x9yBaVIBaSKt5bTYbwy9tmT6pMrGqJKQMVBP5PhBQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Date = _t, Name = _t, Code = _t, #"Connection Type" = _t, Country = _t, Calls = _t, Invalid = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Name", type text}, {"Code", type text}, {"Connection Type", type text}, {"Country", type text}, {"Calls", Int64.Type}, {"Invalid", Int64.Type}}),
#"Removed Columns" = Table.RemoveColumns(#"Changed Type",{"Code"}),
#"Grouped Rows" = Table.Group(#"Removed Columns", {"Date", "Name", "Connection Type", "Country"}, {{"Calls", each List.Sum([Calls]), type text}, {"Invalid", each List.Sum([Invalid]), type text}})
in
#"Grouped Rows"
Table.RemoveColumns will remove the Code column and Table.Group will group the data on the specified columns (Date, Name, Connection Type and Country) and aggregate the data, sum in this case (Calls and Invalid).
You can do this using the UI only. In Power Query Editor, right click the title of Code column and select Remove. Then from Transform tab click on the leftmost button Group By and fill it as follows:
I need to sum all values in each row and display them in a calculated column. As I deal with lots of columns in lots of tables, adding something like
CalculatedColumn = 'public table_name'[column1] + 'public table_name'[column2] + ... + 'public table_name'[column528]
is really inefficient. Is there a shorter way of doing this?
Yes, there is. You should "Unpivot other columns" and then "Group By" using the Query Editor.
Suppose this dataset:
item;col1;col2;col3;col4;col5
apple;1;2;3;4;5
orange;1;2;3;5;8
banana;1;2;4;6;8
Load it up, and open the query editor.
Choose "Unpivot Other Columns":
You should now see this:
On the "Transform" tab in the ribbon, choose the leftmost "Group By" option. And fill out the dialog like so:
You should now have the wanted end result:
You could also skip the Group By step and let your visualization handle that.
PS. Should you need a few non-summed columns too I recommend either creating a duplicate dataset with the same source and either linking it to the original table with a relationship, or merging it so you get a final table with all wanted columns.
Footnote, this is the Power Query that is generated for you:
let
Source = Csv.Document(File.Contents("D:\Experiments\PowerBi\denormalized.csv"),[Delimiter=";", Columns=6, Encoding=1252, QuoteStyle=QuoteStyle.None]),
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"item", type text}, {"col1", Int64.Type}, {"col2", Int64.Type}, {"col3", Int64.Type}, {"col4", Int64.Type}, {"col5", Int64.Type}}),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Changed Type", {"item"}, "Attribute", "Value"),
#"Grouped Rows" = Table.Group(#"Unpivoted Other Columns", {"item"}, {{"SumCol", each List.Sum([Value]), type number}})
in
#"Grouped Rows"
I'm an Excel VBA programmer who's recently discovered Power Query and Power BI.
Often, I will receive requests to change something like:
Into a sort of un-pivoted version that looks like:
This can absolutely be done in VBA, but watching several Power Query videos (and now ordered three books!) It seems it would be much more efficient to do it in Power Query.
I get the over-all idea (create a table from the delimited list, for one) but don't know the language of the actual steps. And I really want to learn these new tools.
Can I buy a vowel? Phone a friend?
Import the table into Power Query using From Table
Click on the second column and choose "Split Column | By Delimiter" from the ribbon. Choose Comma as the delimiter and click OK.
Right click on the first column and choose "Unpivot Other Columns".
Click on the Attribute column and click on "Remove Columns" from the ribbon.
If you go to the Advanced Editor, you can see the code used to generate this:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}, {"Column2", type text}}),
#"Split Column by Delimiter" = Table.SplitColumn(#"Changed Type","Column2",Splitter.SplitTextByDelimiter(",", QuoteStyle.Csv),{"Column2.1", "Column2.2", "Column2.3"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"Column2.1", type text}, {"Column2.2", type text}, {"Column2.3", type text}}),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Changed Type1", {"Column1"}, "Attribute", "Value"),
#"Removed Columns" = Table.RemoveColumns(#"Unpivoted Other Columns",{"Attribute"})
in
#"Removed Columns"