I have a column for vehicle model years. It includes data with hyphens (e.g. 2016 - 2020). I would like to change it to a comma delimited list (2016, 2017, 2018, 2019, 2020). Ultimately, I will split it by the delimiter and unpivot all the columns (or split it by rows). I can't find anything on Google to help with that. Any help is appreciated.
The following code starts with the hyphenated start/end years (don't sweat the 'Source' definition...just a table entered directly into PowerQuery editor).
The key steps are splitting the years on the dash and then generating a list based on the start and end years thus separated with the List.Generate function (#"List of Years" line). The magic statement is:
each let start = [StartYear], end = [EndYear] in List.Generate(() =>
start, each _ <= end, each _ + 1)
The 'let start = [StartYear], end = [EndYear]' is used to copy the value of the field for the respective row under consideration into the variables 'start' and 'end'. This is because the .Generate function is not capable of directly incorporating a field reference.
The .Generate function then runs very much like a 'for' loop to create a list of values: initial value is 'start', continue iterating until the list value is less than or equal to 'end', increment by 1.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMjIwNNM1MjAyUIqNBQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t]),
#"Renamed Columns" = Table.RenameColumns(Source,{{"Column1", "HumanData"}}),
#"HumanData Type" = Table.TransformColumnTypes(#"Renamed Columns",{{"HumanData", type text}}),
#"Split Years" = Table.SplitColumn(#"HumanData Type", "HumanData", Splitter.SplitTextByDelimiter("-", QuoteStyle.Csv), {"StartYear", "EndYear"}),
#"Year Types" = Table.TransformColumnTypes(#"Split Years",{{"StartYear", Int64.Type}, {"EndYear", Int64.Type}}),
#"List of Years" = Table.AddColumn(#"Year Types", "ListOfYears", each let start = [StartYear], end = [EndYear] in List.Generate(() => start, each _ <= end, each _ + 1))
in
#"List of Years"
Once in the list form, you have flexibility in how to expand in PowerQuery.
Hopefully this helps :).
Related
Hey, so a novice here. I dont want the data to be leaked so I have hidden it and marked it with tags though.
Basically what you are looking at is the accuracy data of a classification model. As you can see, for some classes like "A" it predicts all data points correctly, thus it is flagged as True, but we dont have a False column for A because there are no False values. For some reason when we created this table in DAX by grouping a couple of columns and performing the count operation on different table,DAX didnt show False for any values that were completely True.
On the other hand, for "D", 185 values are true and 2 values are false.
We need to showcase this as like a bar chart with maybe a slicer for the different categories. For that we first need the "False" attribute for the values that predicted completely true, then we need to compute the percentage of False and True predictions for each class and then we need to show that in a bar chart.
We are thinking we should pivot the "FLAG" and the "Predicted Values" column for this, and then we will be able to do all the things aforementioned. But because this table was created using DAX from another table, we are unable to find a way to pivot it. It wont evens how up in the Power Query editor. Any tips or help are welcomed. Sorry again, Im a novice so just learning as I work..
You don't need to pivot. You need a flag dimension table joined to your existing table on flag. The dimension contains just two values for true and false and the cardinality is one-to-many unidirectional. Then on your bar chart, make sure to check show items with no data in the field well.
Starting with : (Using enter data and copy paste from excel)
Adding the following code in the advanced editor under Transforme Data:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WSlTSUQoJCnUFUkbmJsYWSrE60UpJCEFDAzOwUDJCyMTUFCyUgqTKAiHk5ugTDDYOLJKKpMjMBCyUhmy6BVwIps8QLJKOpMjUHCyUgSRkbAJXBdNnAnFoJpIQxFFZqI6KBQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [category = _t, Flag = _t, Value = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"category", type text}, {"Flag", type logical}, {"Value", Int64.Type}}),
#"Pivoted Column" = Table.Pivot(Table.TransformColumnTypes(#"Changed Type", {{"Flag", type text}}, "en-SE"), List.Distinct(Table.TransformColumnTypes(#"Changed Type", {{"Flag", type text}}, "en-SE")[Flag]), "Flag", "Value", List.Sum)
in
#"Pivoted Column"
Gives:
Which is what I assume you are after
my data number came in text form, for example: 4.1M 1.22B 3K
Is there a way to turn them back into numbers like 4100000 for 4.1M
Thanks in advance
If you are using card visualization you have feature to change count for thousands, millions etc. It depend on visualization.
You can follow the following steps:
Select the visual where you want to change this notation
Click on the paint roller in the Visualisations pane
You have already activated labels, expand this part
Here you can choose how to show values, select whatever you want
Keep in mind however that big numbers are hard to read and easy to read wrong. It is worth considering to keep them as 4.1M and 1.22B since it keeps them nice and compact. This is especially the case if you use visualisations that grow over time, where overlap might occur easily if you have written-out numbers.
Also, refer to the following documentation for explanation on the Microsoft website:
https://learn.microsoft.com/en-us/power-bi/visuals/power-bi-visualization-customize-title-background-and-legend
Here is an example of how to clean data in Power Query. It is a bit cumbersome, and personally I would contact whoever is responsible for my data source and ask if I could get data in a better format.
The idea is to first figure out what type of multiplier you want to apply to each row, then clean out the "foreign" symbols in the Sample Value column - this is generated using a nifty trick with Character.FromNumber, see link: https://www.excelguru.ca/blog/2015/11/19/keep-only-numbers-in-power-query/ )after that it is simply to convert the original column to a number (might have to remove #"Replaced Value" step depending on your locale) and create a final column that multiplies the number with the multiplier.
You can paste the below into a blank query to see for yourself:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMtEz9FWK1YlWMtQzMnICs4y9IQLGSrGxAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Sample Values" = _t]),
#"Added Conditional Column" = Table.AddColumn(Source, "Custom", each if Text.Contains([Sample Values], "B") then Number.Power(10,9) else if Text.Contains([Sample Values], "M") then Number.Power(10,6) else if Text.Contains([Sample Values], "K") then Number.Power(10,3) else 1),
CharsToRemove = List.Transform({33..45,47,58..126}, each Character.FromNumber(_)),
#"Added Custom" = Table.AddColumn(#"Added Conditional Column", "Result", each Text.Remove([Sample Values],CharsToRemove)),
#"Replaced Value" = Table.ReplaceValue(#"Added Custom",".",",",Replacer.ReplaceText,{"Result"}),
#"Changed Type" = Table.TransformColumnTypes(#"Replaced Value",{{"Result", type number}}),
#"Added Custom1" = Table.AddColumn(#"Changed Type", "Result Value", each [Result]*[Custom], type number),
#"Removed Other Columns" = Table.SelectColumns(#"Added Custom1",{"Result Value"})
in
#"Removed Other Columns"
Need some DAX help with the below data.
I have Columns Uniq ID and Controls.
I want to split the Controls column as shown in New Column.
The string will always start with "AON" and a total of 7 characters (i.e AON0913)
Thanks for help
Here is the Power Query solution. I added comments to understand what each step is doing.
Input Table:
Uniq ID
Controls
RIS123
{AON0913: test 1}, {AON0S18: Safety glasses complying to AS/NZS 1337.1:2010}, {AON0937: Wearing of protective gloves}, {AON0938: First Aid }
RIS345
{AON0913: test 1}, {AON0937: Wearing of protective gloves}, {AON0939: Some SCO's may already have up-to-date vaccinations. }, {AON0938: First Aid }
RIS3223
{AON0S18: Safety glasses complying to AS/NZS 1337.1:2010}, {AON0937: Wearing of protective gloves}, {AON0928: Safety and Compliance uniform Specifications 12-2017 }
RIS0456
{AON0912: test }, {AON0941: controlling test 4321234 }
Power Query M Code:
let
Source = Excel.CurrentWorkbook(){[Name="Table2"]}[Content],
//Add an index so we know the original amount of rows
#"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1, Int64.Type),
//Add a column to create a list of Split strings after ":", the trailing delimiter. This will create a list with multiple rows.
#"Add Column" = Table.AddColumn(#"Added Index", "Split Colon", each Text.Split([Controls],":")),
//Expand the new rows
#"Expanded Split Colon" = Table.ExpandListColumn(#"Add Column", "Split Colon"),
//Filter out any Row that does not contain "AON0"
#"Filtered Rows" = Table.SelectRows(#"Expanded Split Colon", each Text.Contains([Split Colon], "AON0")),
//Extract the text after "AON0", which will be the three digits. Prefix it with "AON0" to create the full string. This is in case it does not have a leading "{" delimiter.
#"Extracted Text After Delimiter" = Table.TransformColumns(#"Filtered Rows", {{"Split Colon", each "AON0" & Text.AfterDelimiter(_, "AON0"), type text}}),
//Group by the Index that we created earlier. We will group all existing columns together as well as create a new column with a comma delimited string for our Output
#"Grouped Rows" = Table.Group(#"Extracted Text After Delimiter", {"Index"}, {{"All Rows", each _, type table [Uniq ID=nullable text, Controls=nullable text, Index=number, Split Colon=text]}, {"Output", each Text.Combine([Split Colon], ", "), type text}}),
//Expand our groups
#"Expanded All Rows" = Table.ExpandTableColumn(#"Grouped Rows", "All Rows", {"Uniq ID", "Controls"}, {"Uniq ID", "Controls"}),
//Remove any duplicated rows
#"Removed Duplicates" = Table.Distinct(#"Expanded All Rows")
in
#"Removed Duplicates"
Output Table:
You can remove the Index column after you remove the duplicates if it is not needed.
Power BI does not have any native functions in the Power query to perform any RegEx operations (which may be most useful here. Please vote ). What you can do is to use build-in option
Column From Examples; You can here type the desired result in a few rows (where you have the longest example - Max AON occurrence. ) and powerbi should indicate what you want (probably the should use multiple time splitbydelimiter + data.combine function + substring)
I have a simple column here.
I am using Power Query to replace some values,
1 becomes 119, 2 becomes 201, 3 becomes 321
There are no particular logic to it, it's to correct the mistyped numbers by users.
I did a find a replace function.
Table.ReplaceValue(#"Changed Type",1,119,Replacer.ReplaceValue,{"Name"}) which worked great.
I want to combine the other find and replace functions into one single step for cleaner code.
I tried this but didn't work.
Table.ReplaceValue(#"Changed Type",{1,2},{119,201},Replacer.ReplaceValue,{"Name"})
I don't want to have 10 different find and replace steps in my PQ just want to have one step.
OPTION-1
Here below is the M code for all replace together-
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlSK1YlWMgKTxkqxsQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t]),
#"AllReplace" = [
#"1" = "119",
#"2" = "201",
#"3" = "321"
],
#"Replaced Value" = Table.TransformColumns(Source,{{"Column1",each Record.FieldOrDefault(AllReplace,_,_)}})
in
#"Replaced Value"
Note: Need to number column as Text. You can convert the column to Number after the Replace Step.
Input
Output
OPTION-2:
You can also create a custom column using the below code-
if [Column1] = 1 then 119 else if [Column1] = 2 then 201 else 321
After creating the custom column, you can keep or remove the source coulmn.
I am trying to find a way in Power BI to transform a table into a specific format for a custom visualization. The transformation that I'd like to do is the following image:
Is this possible to do? I've thought of unpivoting or transposing but haven't seemed to figure out how to get past this point.
I do not really understand the use case of this transformation, but you can achieve what you want by copying the source table, remove all columns except 2, and then append all copies.
Let's say we have one table, named Table, like this:
Make 3 copies of the table, by clicking New Source -> Blank Query:
and enter =Table as query text, where Table is the name of our source table. If our queries are named Query1, Query2 and Query3, then click on each of them and remove the extra column and keep A and B in Query1, keep B and C in Query2, and C and D in Query3. Rename the remaining columns to be named the same way in all 3 queries and click Append Queries or Append Queries as New, then select the 3 copes:
This will give you the result you ask for:
Or you could do it in one go, without creating three different queries. Makes your Queries pane a bit nitter
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUXICYmcgdlGKjQUA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [c1 = _t, c2 = _t, c3 = _t, c4 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"c1", type text}, {"c2", type text}, {"c3", type text}, {"c4", type text}}),
_t1 = Table.SelectColumns(#"Changed Type", {"c1", "c2"}),
_t1_r = Table.RenameColumns(_t1, {{"c1", "ca"}, {"c2", "cb"}}),
_t2 = Table.SelectColumns(#"Changed Type", {"c2", "c3"}),
_t2_r = Table.RenameColumns(_t2, {{"c2", "ca"}, {"c3", "cb"}}),
_t3 = Table.SelectColumns(#"Changed Type", {"c3", "c4"}),
_t3_r = Table.RenameColumns(_t3, {{"c3", "ca"}, {"c4", "cb"}}),
_res = Table.Combine({_t1_r, _t2_r, _t3_r})
in
_res
One use case I can think of this transformation will be useful is that, when we have a time series data like this, and we want to compare the values of current and last periods.
So here is a version for the case when the number of columns is variable.
let
Source = ...,
SingleRow = Table.SingleRow(Source),
Values = Record.FieldValues(SingleRow),
Pairs = List.Zip({List.RemoveLastN(Values, 1), List.RemoveFirstN(Values, 1)}),
Result = Table.FromRows(Pairs, {"Last", "Current"})
in
Result
I am implying the "rules" for this transformation are then you want a table where the first column and second column form each "pair" of items in the original set. Given the data loaded into Power Query like this:
I would transpose the data:
Then make two references to the query. In the first one remove the last row, in the second one remove the first row. Then add an index column to both.
(result of the first reference looks like this)
Then just do a merge between the two references and you get the two column outputs show in the example.