Power Query - Pivoting without gaps - list

Sorry, I could not think of a better, fitting title. The text should clarify ...
In thefollowing scenario: 4 employees should work on different days. The table looks like this:
Date Name
12/13/2018 Carol, John
12/14/2021 Peter, Carol, John, Alice
12/15/2018 Alice
12/16/2018 Peter, Alice
12/17/2018 John, Peter
OK, split names at "," (with trailing space) and paste as lines; since there must be 3 columns: insert index column and then pivoting brings this result:
Index 12/13/2018 12/14/2018 12/15/2018 12/16/2018 12/17/2018
0 Carol null null null null
1 John null null null null
2 null Peter null null null
3 null Carol null null null
4 null John null null null
5 null Alice null null null
6 null null Alice null null
7 null null null Peter null
8 null null null Alice null
9 null null null null John
10 null null null null Peter
But what I want as a result:
Index 12/13/2018 12/14/2018 12/15/2018 12/16/2018 12/17/2018
0 Carol Peter Alice Peter John
1 John Carol null Alice Peter
2 null John null null null
3 null Alice null null null
I will delete the column index later.
So I got to the desired goal: with the exception of the column index and the (correctly arranged) column 12/13/2018 create each additional column as a new query and filter / delete the null cells. Then delete the previous date column and re-insert the newly created date column into the query via merge.
This is very cumbersome and tedious in my opinion. Is there a better way in Power Query?
I would be glad, if you would describe a better way with simple words and if possible step by step (not only M-Code).

Take the table, split by delimiter (commas,each occurance), select then transpose entire table, add index
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"List", type text}}),
#"Split Column by Delimiter" = Table.SplitColumn(#"Changed Type", "List", Splitter.SplitTextByDelimiter(",", QuoteStyle.Csv), {"List.1", "List.2", "List.3", "List.4"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"List.1", type text}, {"List.2", type text}, {"List.3", type text}, {"List.4", type text}}),
#"Transposed Table" = Table.Transpose(#"Changed Type1"),
#"Added Index" = Table.AddIndexColumn(#"Transposed Table", "Index", 0, 1),
#"Reordered Columns" = Table.ReorderColumns(#"Added Index",{"Index", "Column1", "Column2", "Column3", "Column4", "Column5"})
in #"Reordered Columns"

Related

Iterating through a column in a power bi query and storing some of those values in a new column

I have a column These values are lying in sets of 2, representing upper and lower tolerance values (in the same column, unfortunately). They follow the pattern : upper, lower, upper, lower, ... and so on. I would like to loop through this column and print the lower tol values in a new column called lower tol.
There is no specific manner to detect a lower tol value, since both are plain numbers for the machine. However, as the pattern suggests, every alternate value would be a lower tol value. So it can looped through accordingly. Is there a way to do this? If so, how would the M code/ power query code look like? Thank you!
ps. there are also some null values or alpha-numeric valyes lying in this column (not in the picture), which should be avoided by the loop. Thank you.
.
It sounds like you want to (a) remove null rows (b) remove rows where the contents are not numeric (c) separate the remaining rows into two columns from alternating rows
To do this in easy steps:
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Filtered Rows" = Table.SelectRows(Source, each try [Column1] <> null and Number.From([Column1])>0 otherwise false),
#"Added Index" = Table.AddIndexColumn(#"Filtered Rows", "Index", 1, 1, Int64.Type),
#"Inserted Modulo" = Table.AddColumn(#"Added Index", "Modulo", each Number.Mod([Index], 2), type number),
#"Added Custom1" = Table.AddColumn(#"Inserted Modulo", "Custom", each if [Modulo]=1 then [Index] else null),
#"Filled Down" = Table.FillDown(#"Added Custom1",{"Custom"}),
#"Removed Columns" = Table.RemoveColumns(#"Filled Down",{"Index"}),
#"Pivoted Column" = Table.Pivot(Table.TransformColumnTypes(#"Removed Columns", {{"Modulo", type text}}, "en-US"), List.Distinct(Table.TransformColumnTypes(#"Removed Columns", {{"Modulo", type text}}, "en-US")[Modulo]), "Modulo", "Column1", List.Sum),
#"Removed Columns1" = Table.RemoveColumns(#"Pivoted Column",{"Custom"})
in #"Removed Columns1"
Column1
A123
B1234
1
5
A1234
6
13
25
A1234
38

Subtracting two rows from same column with condition on different column in Power BI

I am creating a table in Power Bi, where I have job level and for each job level I have two rows one for "Existing Employees" and second for "New Hires". Existing Employees and new hires have comp ratios. I want to create a new column which provides the difference between New Hire comp ratio and existing employees comp ratio for each job level. If result is negative then show blank else show the result.
Something like below:
Job Level Employee Group Comp Ratio Difference
3 Existing 108% -108 ( don't show this)
3 New Hire 0%
4 Existing 107% 3
4 New Hire 110%
5 Existing 104% -1 (Don't show this)
5 New Hire 103%
Thanks,
CSTech
screenshot below :
You can do this in Power Query M Code
Enter the Power Query Editor Home => Transform Data
Enter the Advanced Editor Home => Query => Advanced Editor
In the editor delete, the last few lines starting with in ...
Paste the code below after the new end of your code
In the first line of newly pasted code, change #"Previous Step" to the name of the actual previous step in your code.
Algorithm
Group into pairs by Job Level
Add a column to each grouped sub table
if the Employee Group is New Hire then write a null
If the 2nd entry in the Comp Ratio column is greater than the first, then subtract and write the difference, otherwise write a 'null'
//Add this to your code
#"Grouped Rows" = Table.Group(#"Previous Step", {"Job Level"}, {
{"Difference", (t)=> Table.AddColumn(t,"Difference", each
if [Employee Group] = "New Hire" then null
else if t[Comp Ratio]{1} > t[Comp Ratio]{0}
then t[Comp Ratio]{1} - t[Comp Ratio]{0}
else null,Percentage.Type), type table[Job Level=Int64.Type, Comp Ratio=Percentage.Type, Difference=Percentage.Type]}
}),
#"Expanded Difference" = Table.ExpandTableColumn(#"Grouped Rows", "Difference",
{"Comp Ratio", "Difference"})
in
#"Expanded Difference"
Results with your data
M Code to reproduce the above table
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlbSUXKtyCwuycxLBzINDSz0DAxUlWJ1IFJ+qeUKHplFqUCmAVzCBF2POYoUkh5DQ4QuU3RdJihSyLoMjCFSsQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Job Level" = _t, #"Employee Group" = _t, #"Comp Ratio" = _t]),
#"Previous Step" = Table.TransformColumnTypes(Source,{
{"Job Level", Int64.Type},
{"Employee Group", type text},
{"Comp Ratio", Percentage.Type}}),
//Add this to your code
#"Grouped Rows" = Table.Group(#"Previous Step", {"Job Level"}, {
{"Difference", (t)=> Table.AddColumn(t,"Difference", each
if [Employee Group] = "New Hire" then null
else if t[Comp Ratio]{1} > t[Comp Ratio]{0}
then t[Comp Ratio]{1} - t[Comp Ratio]{0}
else null,Percentage.Type), type table[Job Level=Int64.Type, Comp Ratio=Percentage.Type, Difference=Percentage.Type]}
}),
#"Expanded Difference" = Table.ExpandTableColumn(#"Grouped Rows", "Difference",
{"Comp Ratio", "Difference"})
in
#"Expanded Difference"

how to create a DAX measure / power query process / which returns a value from another row, which has the same date & time

Please find below links to the Fact table and the overview of all tables. I would like to create a DAX measure, or new column in the Fact table ("transactions"), where:
The currency is NOT equal to EUR (e.g. "BTC"),
Giving the value equal to a opposite value of "amount", as given in the row with EUR as currency (e.g. +5. Positive for negative and vice versa),
Where the date and time of the two rows (EUR and non-EUR) have the same values (e.g. 03/11/2021 and 12:28:06)
The "type" = "trade",
In all other cases, I think it would be best to give a value of 0.
In my Fact table screenshot, I manually added the EUR_amt column in Excel to show what I would like to create
I think it's also possible to add the column, then group by time and date, such that the rows with EUR as currency with EUR_amt being 0, would be removed. All using power query. That would be even better.
(The "Currencies" table just uses the distinct values of the "currency" column in the "transactions" table, via PowerQuery. Not relevant for this question I think)
Many thanks in advance!
-YK
Fact table "transactions"
Overview of tables
Calculated Column:
EUR_amt =
IF (
OR ( transactions[type] <> "trade", transactions[currency] = "EUR" ),
0,
- LOOKUPVALUE (
transactions[amount],
transactions[Date], transactions[Date],
transactions[Time], transactions[Time],
transactions[currency], "EUR",
0
)
)
Here's one way to do this using just Power Query and the Advanced Editor
Group by data and time
Generate a custom column for each subtable based on your rules
Expand the subtables, remove those with "0", and re-order the columns
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"type", type text}, {"currency", type text}, {"Date", type date}, {"Time", type time}, {"amount", type number}}),
//Group by Date and Time
#"Grouped Rows" = Table.Group(#"Changed Type", {"Date", "Time"}, {
//Add the Eur amt column to the grouped table
{"EUR_amt", (t)=>let
//determine relevant euro amt for each sub table
eur= t[amount]{List.PositionOf(t[currency],"EUR")},
//add the column to each subtable basaed on conditions
addCol = Table.AddColumn(t, "EUR_amt",
each if [type]= "trade" and [currency]<>"EUR" then -eur else 0)
in
addCol}}),
//Expand the new table
//Filter out the 0's
//reorder the columns
#"Expanded EUR_amt" = Table.ExpandTableColumn(#"Grouped Rows", "EUR_amt", {"type", "currency", "amount", "EUR_amt"}, {"type", "currency", "amount", "EUR_amt"}),
#"Filtered Rows" = Table.SelectRows(#"Expanded EUR_amt", each ([EUR_amt] <> 0)),
#"Reordered Columns" = Table.ReorderColumns(#"Filtered Rows",{"type", "currency", "Date", "Time", "amount", "EUR_amt"})
in
#"Reordered Columns"

How to erase rows with combined criteria?

How to erase rows of a table with two of criteria of two columns, combined? For example, I want to erase rows of products that come from France (country column) AND got the word ''wood'' in their name (name column).
Add column, custom column with formula
= if [Country]="France" and [Name]="Wood" then 1 else 0
Then use filter drop down atop that new column to uncheck the selection for [ ] 1 while leaving [x] 0
Potentially you meant that the Name field would contain the letters wood within a larger string of text (like Woodland), so you could use instead
= if [Country]="France" and Text.Contains(Text.Lower([Name]),"wood") then 1 else 0
You can use the Table.SelectRows method:
let
Source = Excel.CurrentWorkbook(){[Name="Table10"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,
{{"Country", type text}, {"Name", type text}, {"Value", type text}}),
filtered = Table.SelectRows(#"Changed Type",
each not ([Country]="France" and Text.Contains([Name],"Wood",Comparer.OrdinalIgnoreCase)))
in
filtered

How to remove duplicate values in one column and concatenate values from another column where removal happens?

The Data:
ACCOUNT DESC
Gallup 1
Gallup 2
Phoenix 2
Red Rock 1
Red Rock 2
Albuquerque 1
The desired output:
ACCOUNT DESC
Gallup 1,2
Phoenix 2
Red Rock 1,2
Albuquerque 1
but in a general scope as this is a small subset (Many other accounts, 100+)
Is there a way to remove duplicate values of "ACCOUNT" and to concatenate values from the removed duplicates where the "ACCOUNT" matched?? (this method is the preferred approach desired if possible)
Right click Account column, Group By
use New Column name : Data, Operation: All Rows,
Add Column, Custom Column, formula
=Table.Column([Data],"DESC")
Click on arrows at top of new column, extract value, use comma delimiter
Remove extra column
Full sample code if data was in Table1:
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Grouped Rows" = Table.Group(Source, {"ACCOUNT"}, {{"Data", each _, type table}}),
#"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each Table.Column([Data],"DESC")),
#"Extracted Values" = Table.TransformColumns(#"Added Custom", {"Custom", each Text.Combine(List.Transform(_, Text.From), ","), type text}),
#"Removed Columns" = Table.RemoveColumns(#"Extracted Values",{"Data"})
in #"Removed Columns"