Hi need to add 1000+ calculated columns in power bi which provide the count per entry, for example.*means calculated columns
ID
RankCode
Count_RankCode*
RankAdvance
Count_RanAdvance*
1000
AAA
2
XYZ
2
1001
AAA
2
XYA
1
1002
AAB
1
XYZ
2
found the right way to count in power BI DAX:
COUNTROWS(FILTER('24Jun_1973',[rankCode]=earlier([rankCode])))
Requirement:
add 1000 columns that count rows in probably in one code using DAX
or create the 1000 count cloumn in power query M language (need it to be fast since raw date is 60gb).
As suggested by #smpa01, I was able to complete this task using the tabular editor. Just used the DAX script in tabular editor, put my all my measures in there since I was able to create all expressio in excels as it is just repeating then voila, 1000 measures added.
example:
Measure '24Jun_1973'[measure]=calculate(COUNTROWS(FILTER('28Jun_1973',[rankcode]='28Jun_1973'[rankcode])))
Measure '24Jun_1973'[measure2]=calculate(COUNTROWS(FILTER('28Jun_1973',[rankAdvance]='28Jun_1973'[rankAdvance])))
I have no idea why you would want another 1000 columns.
If you really want to though, in powerquery, you could unpivot, group and count, append the results to original data, then re-pivot. I don't know how fast it would be. I suspect not very.
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}, {"RankCode", type text}, {"RankAdvance", type text}}),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Changed Type", {"ID"}, "Attribute", "Value"),
// group and count
#"Grouped Rows" = Table.Group(#"Unpivoted Other Columns", {"Attribute", "Value"}, {{"Count", each Table.RowCount(_), type number}}),
#"Duplicated Column" = Table.DuplicateColumn(#"Grouped Rows", "Attribute", "Attribute - Copy"),
#"Change column name" = Table.TransformColumns(#"Duplicated Column",{{"Attribute - Copy", each "count_" & _, type text}}),
// append back to original table, then repivot
#"Merged Queries" = Table.NestedJoin(#"Unpivoted Other Columns",{"Attribute", "Value"},#"Change column name",{"Attribute", "Value"},"Table2",JoinKind.LeftOuter),
#"Expanded Table1" = Table.ExpandTableColumn(#"Merged Queries", "Table2", {"Count", "Attribute - Copy"}, {"Count", "Attribute - Copy"}),
#"Removed Columns" = Table.RemoveColumns(#"Expanded Table1",{"Attribute", "Value"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Count", "Value"}, {"Attribute - Copy", "Attribute"}}),
combined = #"Unpivoted Other Columns" & #"Renamed Columns",
#"Pivoted Column" = Table.Pivot(combined, List.Distinct(combined[Attribute]), "Attribute", "Value")
in #"Pivoted Column"
Related
I have a bit complicated PowerQuery query which has many steps. Within these steps I have a date, team and conditions as below (not actual data)
So the challenge is that I want to count the Pass number for each team for each day and then create another count for Pass and Fail and then it will be used in so many calculations which I can handle later.
I have tried many options, like for example grouping, but it was so confusing because as I mentioned before, the query has so many columns and calculations now. I could successfully solve the issue by creating DAX measure, but the issue here that I need to calculate the average outcome which is not possible to because I couldn't also average the measure of the outcome. So I have no other option but to make the countif though PowerQuery.
Appreciate your help and ideas.
Raw data as text is here in google sheets
I am assuming that your dates showing the year 0203 is a typo and should be 2023
Not sure exactly what you want for output, but you should be able to adapt the below.
The solution seems to be a simple grouping with a count of the number of passes and/or fails.
The below code generates a separate column for passes and fails per team and date.
It is not sorted in the original order, but that could be added if necessary.
#"Grouped Rows" = Table.Group(#"Previous Step", {"Date", "Team"}, {
{"Pass", (t)=>List.Count(List.Select(t[#"PASS/FAIL"], each _ = "Pass")), Int64.Type},
{"Fail", (t)=>List.Count(List.Select(t[#"PASS/FAIL"], each _ = "Fail")), Int64.Type}
})
Using your data table from the Google sheet (after correcting the year):
let
Source = Excel.CurrentWorkbook(){[Name="Table12"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Team", type text}, {"PASS/FAIL", type text}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"Date", "Team"}, {
{"Pass", (t)=>List.Count(List.Select(t[#"PASS/FAIL"], each _ = "Pass")), Int64.Type},
{"Fail", (t)=>List.Count(List.Select(t[#"PASS/FAIL"], each _ = "Fail")), Int64.Type}
})
in
#"Grouped Rows"
Note If you require that all teams show on all dates even if they didn't play, one merely creates a new table containing all dates and teams; and then Joins that to the original table, as in the code below:
let
Source = Excel.CurrentWorkbook(){[Name="Table12"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Team", type text}, {"PASS/FAIL", type text}}),
//If you need to show all teams on all dates, even if they didn't play on a date
//we merely create a blank table (dates and teams), and execute an outer join.
//then remove the original date/team columns before the grouping.
#"Date List" = List.Distinct(#"Changed Type"[Date]),
#"All Teams" = List.Distinct(#"Changed Type"[Team]),
Blank = Table.FromColumns(
{List.Combine(List.Transform(#"Date List", each List.Repeat({_}, List.Count(#"All Teams")))),
List.Repeat(#"All Teams", List.Count(#"Date List"))},
type table[dates=date, teams=text]),
join = Table.Join(Blank,{"dates","teams"}, #"Changed Type",{"Date","Team"}, JoinKind.LeftOuter),
#"Removed Columns" = Table.RemoveColumns(join,{"Date", "Team"}),
#"Grouped Rows" = Table.Group(#"Removed Columns", {"dates", "teams"}, {
{"Pass", (t)=>List.Count(List.Select(t[#"PASS/FAIL"], each _ = "Pass")), Int64.Type},
{"Fail", (t)=>List.Count(List.Select(t[#"PASS/FAIL"], each _ = "Fail")), Int64.Type}
}),
#"Sorted Rows" = Table.Sort(#"Grouped Rows",{{"dates", Order.Ascending}, {"teams", Order.Ascending}})
in
#"Sorted Rows"
If you need the zeroes in your final output, you'll need to do a cross join to bring in combinations not present in the original.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMtQ31DcyMDJW0lFyDAISAYnFxUqxOigSrn44JFxcgYRbYmYOuoSfD7KEkb4RilEkSeA0Cr8E3LlIEmDnEpYw1jfWNzAywDSKIgmcduCUQI0PJAnU+CDGKNSwotwOiFGxAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Date = _t, Team = _t, #"PASS/FAIL" = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type text}, {"Team", type text}, {"PASS/FAIL", type text}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"Date", "Team"}, {{"Count", each Table.RowCount(_), Int64.Type}}),
Custom1 = List.Distinct( #"Grouped Rows"[Date]),
#"Converted to Table" = Table.FromList(Custom1, Splitter.SplitByNothing(), {"Date"}, null, ExtraValues.Error),
#"Added Custom" = Table.AddColumn(#"Converted to Table", "Team", each List.Distinct( #"Grouped Rows"[Team])),
#"Expanded Team" = Table.ExpandListColumn(#"Added Custom", "Team"),
#"Merged Queries" = Table.NestedJoin(#"Expanded Team", {"Date", "Team"}, #"Grouped Rows", {"Date", "Team"}, "Expanded Team", JoinKind.LeftOuter),
#"Expanded Expanded Team" = Table.ExpandTableColumn(#"Merged Queries", "Expanded Team", {"Count"}, {"Count"}),
#"Replaced Value" = Table.ReplaceValue(#"Expanded Expanded Team",null,0,Replacer.ReplaceValue,{"Count"})
in
#"Replaced Value"
Stepping through the code:
Group by date and team and create a count:
Get a distinct list of dates
Convert to table
Add column for a cross join with all teams (to get zero values later)
Expand
Merge back to the grouped step to pull in the previous grouped values.
Replace nulls with 0
You can amend this for your other question by simply filtering for pass before you do the grouping.
Just add column, custom column
= 1
then click select Pass/Fail column, transform .. pivot .. and choose the new column as values column
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom", each 1),
#"Pivoted Column" = Table.Pivot(#"Added Custom", List.Distinct(#"Added Custom"[PassFail]), "PassFail", "Custom", List.Sum)
in #"Pivoted Column"
I'm trying to count and sum based on multiple columns. When I tried the "group by" function in transform data, it gives me a timeout error.
Below is an illustrative example of what I'm trying to do. In the real dataset, the number of columns is 30+ and the number of possible entries in each column is also large, resulting in many unique combinations.
I'm not sure if there are other functionalities in Power BI that can achieve this, please send help!
Have this:
Want this:
Im going to go ahead and guess that perhaps you want to sum all your columns without knowing how many of them you have, and this code can work on any number of columns, grouping the first two
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(Source, {"Column1", "Column2"}, "Attribute", "Value"),
#"Grouped Rows" = Table.Group(#"Unpivoted Other Columns", {"Column1", "Column2", "Attribute"}, {{"Count", each Table.RowCount(_), Int64.Type}, {"SUM", each List.Sum([Value]), type number}}),
#"Pivoted Column" = Table.Pivot(#"Grouped Rows", List.Distinct(#"Grouped Rows"[Attribute]), "Attribute", "SUM", List.Sum)
in #"Pivoted Column"
In Power Query, on the Tranform ribbon, click "Group By" and enter these settings.
I need some help:
I want to scale the first table using the factors of the second table to get the third table:
The indexes in the first table are repeated (you can think of them as categories).
How can I get the third table using only the Power Query Editor of POWER BI Desktop?
This is the pbi
Thank you!
(Your sample data is wrong, with the index for the Values to Scale not matching that of Scaled Table, but anyway)
Powerquery method using unpivot, merge and then pivot. This is the code for the Values to Scale data, to convert it to be Scaled Table using data from FACTORS query
let Source = Excel.CurrentWorkbook(){[Name="ValuestoScaleData"]}[Content],
// assumes presence of query FACTORS that contains data as shown
FactorTable1 = Table.UnpivotOtherColumns(FACTORS, {"INDEX"}, "Attribute", "Value"),
FactorTable2 = Table.ReplaceValue(FactorTable1,"FACTOR ","",Replacer.ReplaceText,{"Attribute"}),
#"Added Index" = Table.AddIndexColumn(Source, "Index.1", 0, 1, Int64.Type),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Added Index", {"INDEX", "Index.1"}, "Attribute", "Value"),
#"Replaced Value" = Table.ReplaceValue(#"Unpivoted Other Columns","VALUE ","",Replacer.ReplaceText,{"Attribute"}),
#"Merged Queries" = Table.NestedJoin(#"Replaced Value", {"INDEX", "Attribute"}, FactorTable2, {"INDEX", "Attribute"}, "Table1", JoinKind.LeftOuter),
#"Expanded Table2" = Table.ExpandTableColumn(#"Merged Queries", "Table1", {"Value"}, {"Value.1"}),
#"Added Custom" = Table.AddColumn(#"Expanded Table2", "MULT", each [Value]*[Value.1]),
Rename = Table.TransformColumns(#"Added Custom",{{"Attribute", each "Scaled "&_, type text}}),
#"Removed Columns" = Table.RemoveColumns(Rename,{"Value", "Value.1"}),
#"Pivoted Column" = Table.Pivot(#"Removed Columns", List.Distinct(#"Removed Columns"[Attribute]), "Attribute", "MULT", List.Sum),
#"Sorted Rows" = Table.Sort(#"Pivoted Column",{{"Index.1", Order.Ascending}}),
#"Removed Columns1" = Table.RemoveColumns(#"Sorted Rows",{"Index.1"})
in #"Removed Columns1"
I have Table A with data of specific people doing their tasks like this:
I have Table B with data of needs for specific people for different periods of time like this:
I also have additional table C with period definitions:
Period no | Date from | Date to
--------------------------------------
1 | 27/01/2021 | 24/02/2021
2 | 25/02/2021 | 24/03/2021
...
There are 2 problems here:
Someone in Table A can have Start and End dates spanning multiple periods, like for example Human B
The Start and End dates may not encompass whole Periods, they can be for example just for a couple of days. And so there's an algorithm that calculates whether this counts as a period or not:
if this is less than 5 days, than it doesn't count
if this is between 6 and 14 days, than it's 0.5 period
if it's more than 14 days, than it's 1 period
So now I want to merge Table A with Table B, to compare needs with what was delivered, for every period. The question is how to go with this?
My first thought was to add columns to Table A for Period and Quantity, to be able to group & merge over it - but what about when this deployment can span over multiple periods? Also how to implement this conditional logic for periods?
I think this works
Pull in Period definitions as Table1
Add a custom column using formula
= {Number.From([Date from])..Number.From([Date to])}
And then expand that to rows. That gives you a match for every date to every period
File .. Close and Load ... Connection
Full sample code for that part is:
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Period no", Int64.Type}, {"Date from", type date}, {"Date to", type date}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each {Number.From([Date from])..Number.From([Date to])}),
#"Expanded Custom" = Table.ExpandListColumn(#"Added Custom", "Custom"),
#"Removed Columns" = Table.RemoveColumns(#"Expanded Custom",{"Date from", "Date to"})
in #"Removed Columns"
Pull in your TableA, called Table2 here
Add custom column with similar formula and expand to rows
= {Number.From([Start of Deployment]) .. Number.From([End of Deployment])}
Now merge the other table into this one and pull in period
Click select the type and period columns and group them, pulling in the maximum and minimum dates from the new custom column
Add custom column for working duration with formula
= 1+[DayMax]-[DayMin]
Then add a custom column to apply your algo
= if [Duration]<6 then 0 else if [Duration] <15 then 0.5 else 1
Remove extra columns. Done. File ... Close and Load ... Connection
You can merge this back into your Table B as needed
Full code for this table
let Source = Excel.CurrentWorkbook(){[Name="Table2"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Type", type text}, {"Start of Deployment", type date}, {"End of Deployment", type date}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each {Number.From([Start of Deployment]) .. Number.From([End of Deployment])}),
#"Expanded Custom" = Table.ExpandListColumn(#"Added Custom", "Custom"),
#"Removed Columns" = Table.RemoveColumns(#"Expanded Custom",{"Start of Deployment", "End of Deployment"}),
#"Merged Queries" = Table.NestedJoin(#"Removed Columns",{"Custom"},Table1,{"Custom"},"Table1",JoinKind.LeftOuter),
#"Expanded Table1" = Table.ExpandTableColumn(#"Merged Queries", "Table1", {"Period no"}, {"Period no"}),
#"Grouped Rows" = Table.Group(#"Expanded Table1", {"Type", "Period no"}, {{"DayMin", each List.Min([Custom]), type number}, {"DayMax", each List.Max([Custom]), type number}}),
#"Added Custom1" = Table.AddColumn(#"Grouped Rows", "Duration", each 1+[DayMax]-[DayMin]),
#"Added Custom2" = Table.AddColumn(#"Added Custom1", "Algo", each if [Duration]<6 then 0 else if [Duration] <15 then 0.5 else 1),
#"Removed Columns1" = Table.RemoveColumns(#"Added Custom2",{"DayMin", "DayMax", "Duration"})
in #"Removed Columns1"
I need to sum all values in each row and display them in a calculated column. As I deal with lots of columns in lots of tables, adding something like
CalculatedColumn = 'public table_name'[column1] + 'public table_name'[column2] + ... + 'public table_name'[column528]
is really inefficient. Is there a shorter way of doing this?
Yes, there is. You should "Unpivot other columns" and then "Group By" using the Query Editor.
Suppose this dataset:
item;col1;col2;col3;col4;col5
apple;1;2;3;4;5
orange;1;2;3;5;8
banana;1;2;4;6;8
Load it up, and open the query editor.
Choose "Unpivot Other Columns":
You should now see this:
On the "Transform" tab in the ribbon, choose the leftmost "Group By" option. And fill out the dialog like so:
You should now have the wanted end result:
You could also skip the Group By step and let your visualization handle that.
PS. Should you need a few non-summed columns too I recommend either creating a duplicate dataset with the same source and either linking it to the original table with a relationship, or merging it so you get a final table with all wanted columns.
Footnote, this is the Power Query that is generated for you:
let
Source = Csv.Document(File.Contents("D:\Experiments\PowerBi\denormalized.csv"),[Delimiter=";", Columns=6, Encoding=1252, QuoteStyle=QuoteStyle.None]),
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"item", type text}, {"col1", Int64.Type}, {"col2", Int64.Type}, {"col3", Int64.Type}, {"col4", Int64.Type}, {"col5", Int64.Type}}),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Changed Type", {"item"}, "Attribute", "Value"),
#"Grouped Rows" = Table.Group(#"Unpivoted Other Columns", {"item"}, {{"SumCol", each List.Sum([Value]), type number}})
in
#"Grouped Rows"