Power BI - How to create another table from original source - powerbi

I wrote a query that pulls data into Power BI. I was wondering if I can create another query that pulls in the original data without certain columns. I know that I can delete a column but I was wondering if I can remove a column and have other columns aggregated. I want to do this from the back-end (PowerQuery). I know I can create another query without including the other column but since this is real-time data, I need to pull the data from the original query.
This is the original data.
This is what I am trying to achieve. I want to remove the column 'Code' but as well as having the other columns aggregated (Calls, Invalid) and distinct columns (Date, Name, Connection Type).
Is this possible on the power query?!

Of course it is possible. Here is an example M code how to do that in Power Query:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("ndKxCoMwEAbgd8ksohdN5y527VDoIA5BQwnYCufSx28oKZHmzqQdJEL4uPvv0vcCoDyUUEElCnExT726s3bf1aKZ3Hm8G7Sjdn/yfTMUtAHSNKQ5mQVvVrOV2oT6r5b0ajbrmlHurNF+B+tQP0bj++aJzCfdvKCdtqGi9iAB2Vz0wgJsyDFu+qz5pxEV22dsuH0myQ4lacKi2x9yBaVIBaSKt5bTYbwy9tmT6pMrGqJKQMVBP5PhBQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Date = _t, Name = _t, Code = _t, #"Connection Type" = _t, Country = _t, Calls = _t, Invalid = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Name", type text}, {"Code", type text}, {"Connection Type", type text}, {"Country", type text}, {"Calls", Int64.Type}, {"Invalid", Int64.Type}}),
#"Removed Columns" = Table.RemoveColumns(#"Changed Type",{"Code"}),
#"Grouped Rows" = Table.Group(#"Removed Columns", {"Date", "Name", "Connection Type", "Country"}, {{"Calls", each List.Sum([Calls]), type text}, {"Invalid", each List.Sum([Invalid]), type text}})
in
#"Grouped Rows"
Table.RemoveColumns will remove the Code column and Table.Group will group the data on the specified columns (Date, Name, Connection Type and Country) and aggregate the data, sum in this case (Calls and Invalid).
You can do this using the UI only. In Power Query Editor, right click the title of Code column and select Remove. Then from Transform tab click on the leftmost button Group By and fill it as follows:

Related

Remove Duplicated strings from a cell in Power Query for Power BI

I'm trying to remove duplicates from two cells, and it seems a nowhere close a solution. So, I would like to ask for your assistance.
I have a table with a few tickets (TKS00XX) and their corresponding causing records (CSR00XX), and additionally to that I also have the investigation records (INV00XX) linked to the tickets. What happens is that in the investigation records we also have the causing records, which sometimes is the causing records linked back to the tickets.
In the example above, it is how is the table at source. In Power BI, I can group the records by the ticket and Investigation Records where I'd get the following
I'd like to merge these two causing records columns and remove the duplicated items (for example, the item highlighted in red (CSR0032). And this is the result I'm trying to get.
Do you know how I can achieve this result in the Power Query Editor for Power BI?
I have tried unpivoting the Causing Record tables and then grouping then by the ticket number ad investigation record numbers, but it removed some entries that shouldn't be removed.
Duplicate your table
From the 1st table remove the 1st CAUSING RECORD
From the 2nd table remove the 2nd CAUSING RECORD
Append both tables as new
Remove duplicate rows
Group by TICKET and INVESTIGATION RECORD and aggregate CAUSING RECORD AS SUM of CAUSING RECORD.
Since the last row will lead to an error, edit the formula in the formula bar and change List.Sum([CAUSING RECORD]) to Text.Combine([CAUSING RECORD], ", ")
The M-Code for the 3 involved tables is
Table1
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WCvEONjAwNFLSUXIODjIwMAKxPP3CgCyYkLGRUqwOhkJjLAqNsSk0NMRQaGKCUGhqCtduClMIFzKzxKYQqB1doamFUmwsAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [TICKET = _t, #"CAUSING RECORD" = _t, #"INVESTIGATION RECORD" = _t, #"CAUSING RECORD.1" = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"TICKET", type text}, {"CAUSING RECORD", type text}, {"INVESTIGATION RECORD", type text}, {"CAUSING RECORD.1", type text}}),
#"Removed Columns" = Table.RemoveColumns(#"Changed Type",{"CAUSING RECORD.1"}),
#"Reordered Columns" = Table.ReorderColumns(#"Removed Columns",{"TICKET", "INVESTIGATION RECORD", "CAUSING RECORD"})
in
#"Reordered Columns"
Table2
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WCvEONjAwNFLSUXIODjIwMAKxPP3CgCyYkLGRUqwOhkJjLAqNMRX6+3t6Yig0MUEoNDWFazeFKYQLmVliUwjUjq7Q1EIpNhYA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [TICKET = _t, #"CAUSING RECORD" = _t, #"INVESTIGATION RECORD" = _t, #"CAUSING RECORD.1" = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"TICKET", type text}, {"CAUSING RECORD", type text}, {"INVESTIGATION RECORD", type text}, {"CAUSING RECORD.1", type text}}),
#"Removed Columns" = Table.RemoveColumns(#"Changed Type",{"CAUSING RECORD"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"CAUSING RECORD.1", "CAUSING RECORD"}})
in
#"Renamed Columns"
Table3
let
Source = Table.Combine({Table1, Table2}),
#"Removed Duplicates" = Table.Distinct(Source),
#"Sorted Rows" = Table.Sort(
#"Removed Duplicates",{{"TICKET", Order.Ascending}}),
#"Grouped Rows" = Table.Group(
#"Sorted Rows",
{"TICKET", "INVESTIGATION RECORD"},
{
{"CAUSING RECORD", each Text.Combine([CAUSING RECORD], ", "), type nullable text}
}
)
in
#"Grouped Rows"
Once you join your two tables, you can Group by TICKET and INVESTIGATION RECORD.
Within the Table.Group function add a custom aggregation of the CAUSING RECORD columns, remove the duplicates, optionally sort, and combine them as a text string.
In the code below, it is the Table.Group function that is important; the rest is just setup and housekeeping.
Tickets
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WCvEONjAwNFLSUXIODjIwMDJSitXBEDXGKmpoiBA1NYWrNcUmamKiFBsLAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [TICKET = _t, #"CAUSING RECORD" = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"TICKET", type text}, {"CAUSING RECORD", type text}})
in
#"Changed Type"
Investigation
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45W8vQLMzAwUtJRcg4OMjAwNlKK1cEQNMYiaGKCEDSFCZpZYhE0tVCKjQUA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"INVESTIGATION RECORD" = _t, #"CAUSING RECORD" = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"INVESTIGATION RECORD", type text}, {"CAUSING RECORD", type text}})
in
#"Changed Type"
Combined
let
//Combine the two tables
#"Append Columns" = Table.FromColumns(
Table.ToColumns(Ticket) & Table.ToColumns(Investigation),
type table[TICKET=text, CAUSING RECORD=text, INVESTIGATION RECORD=text, CAUSING RECORD2=text]),
//Group by TICKET, INVESTIGATION RECORD
#"Grouped Rows" = Table.Group(#"Append Columns", {"TICKET", "INVESTIGATION RECORD"}, {
{"CAUSING RECORDS", each
Text.Combine(
List.Sort(
List.Distinct(
List.Combine({[CAUSING RECORD],[CAUSING RECORD2]}))),
", "), type text}})
in
#"Grouped Rows"

How to convert a whole number(which is minutes) to a time format DD:HH:MM

First of all, thank you for reading this question
So, I have a set of data which is the difference between two date in "minutes" the raw data type is "Whole Number"
I would like to convert it to be "DD:HH:MM", not sure if anyone could help me, thank you so much!
AFAIK, Power BI does not have or recognize a duration type. If you want to display in that format, it will need to be text.
In Power Query you can transform that column to text with code such as:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45W0jU0NDVUitUBskwtTSAMYwhtCSZNzCAcCzBlZAjhGhqbQATMzC3NlWJjAQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Minutes = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Minutes", Int64.Type}}),
#"Transform to Duration" = Table.TransformColumns(#"Changed Type", {"Minutes", each Duration.ToText(#duration(0,0,_,0)), type text})
in
#"Transform to Duration"
Then, in Desktop, format that column as Text
If you need to do calculations on the duration, you may want to add a column instead of transforming the column, or do the calculations in Power Query.
//You can use this
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"duration", Int64.Type}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each [duration]/1440),
#"Changed Type1" = Table.TransformColumnTypes(#"Added Custom",{{"Custom", type duration}})
in
#"Changed Type1"

Convert a single row into multiple rows, depending on values in a specific column in Power Bi

I have rows of data which can have information in multiple columns that I need to extract and convert into an individual row for each.
E.g.
Original table
Headers are:
Product Code | Description | Location 1 | Location 2 | Location 3
and I need to convert it to:
Product Code | Description | Location
Some products will be available in multiple regions.
If a product is available in Germany and France, there may be an DE in the Location 1 column, and an FR in the Location 2 column, while the location 3 column will be blank.
I need to convert it so that there is a single location column with corresponding entries for each region that product had.
Desired output table
Is there a way to automate this in Power Bi?
Select the Code and description columns then
UnpivotOtherColumns
Remove the blank entries
Remove the Attribute column
Not sure how you want your results sorted, but you could easily add a sorting algorithm to below.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUfJILAGSLq5AAoRidaKVjICM4OTEojQg7RYElwVJGQMZ7jn5ZanFQEaoN5KC2FgA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Product Code" = _t, Description = _t, #"Location 1" = _t, #"Location 2" = _t, #"Location 3" = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Product Code", Int64.Type}, {"Description", type text}, {"Location 1", type text}, {"Location 2", type text}, {"Location 3", type text}}),
//Select Product Code and Description Columns
// Then "Unpivot other Columns
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Changed Type",
{"Product Code", "Description"}, "Attribute", "Location"),
//Remove the blank locations and the "Attrubute" column
#"Filtered Rows" = Table.SelectRows(#"Unpivoted Other Columns", each ([Location] <> "")),
#"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"Attribute"})
in
#"Removed Columns"

Unpivot Data and Separate Integers and Strings in Power Bi

Above is my current input.
Below is the desired output.
Currently, i load the data into Power Bi, and using the Power Query Editor I 'unpivot columns' on 'selected columns only'. This results in a column of mixed data types (Integers and strings). Is anyone able to advise an efficient method upon data load how to separate string values and integers?
Below is the code from Advanced Editor
let
Source = Csv.Document(File.Contents("F:\Surveys\dev\pivottest.csv"),[Delimiter=",", Columns=5, Encoding=1252, QuoteStyle=QuoteStyle.None]),
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"responseid", Int64.Type}, {"q1", Int64.Type}, {"q2", type text}, {"q3", Int64.Type}, {"q4", type text}}),
#"Unpivoted Only Selected Columns" = Table.Unpivot(#"Changed Type", {"q1", "q2", "q3", "q4"}, "Attribute", "Value")
in
#"Unpivoted Only Selected Columns"
After you have unpivoted your data, then use the function Value.Is on your Value column. To check if a value is number, it would look something like this:
Value.Is([Column], Int64.Type)
Example,
IsNumber =Value.Is(Value.FromText([ColumnOfMixedValues]), type number)
To check for a text, it would be:
Value.Is([Column],type text)
Example,
IsText =Value.Is(Value.FromText([ColumnOfMixedValues]), type text)
Value.Is returns a boolean true/false, so you can wrap it in an IF.
if Value.Is(AlphaNumeric, type text) then AlphaNumeric else Number.ToText(AlphaNumeric))

How to add custom column index based on date

I have data as such:
I want to split the data in the following format based on date. So going forward the dates can be split and I can use slicer to select the range and get the index value over each selected dates.
Please help on this.
Your pivoted result does not make sense. The first row for example, combines rows #1 and #8 from your table, but there is a prefix 17/22/38 in row #8 which makes the URL different than the one from row #1. Where did this prefix go? How it disappeared and why? And this is also for the other rows, e.g. Contact-us.
But otherwise, Pivot columns is what you need. If your original table looks like this:
Select Date column and click Pivot column command in the ribbon, and you will get a dialog like this:
Select Index as values column. In you case probably it makes no sense to aggregate indexes, so select Don't Aggregate. This will give you a result like this:
Which is as close to your desired result, as possible (considering the issue that I mentioned above).
And here is the M code to reproduce the steps above:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("hdE9CoAwDIDRq0jmUpL0z55FOujsJAge36CQycSl3/IgJF0WoB6xR0bqEGDdLnnPY59ISjCCCViaPJCk7IEszR4o0uKBKm0eaNL6AMZPMOsIA3Rd0wCEfzOIdFFLsN7KEkn/wxL5Pca4AQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [Date = _t, Search_Term = _t, Url = _t, Index = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Search_Term", type text}, {"Url", type text}, {"Index", Int64.Type}}),
#"Pivoted Column" = Table.Pivot(Table.TransformColumnTypes(#"Changed Type", {{"Date", type text}}, "en-US"), List.Distinct(Table.TransformColumnTypes(#"Changed Type", {{"Date", type text}}, "en-US")[Date]), "Date", "Index")
in
#"Pivoted Column"