Hey, so a novice here. I dont want the data to be leaked so I have hidden it and marked it with tags though.
Basically what you are looking at is the accuracy data of a classification model. As you can see, for some classes like "A" it predicts all data points correctly, thus it is flagged as True, but we dont have a False column for A because there are no False values. For some reason when we created this table in DAX by grouping a couple of columns and performing the count operation on different table,DAX didnt show False for any values that were completely True.
On the other hand, for "D", 185 values are true and 2 values are false.
We need to showcase this as like a bar chart with maybe a slicer for the different categories. For that we first need the "False" attribute for the values that predicted completely true, then we need to compute the percentage of False and True predictions for each class and then we need to show that in a bar chart.
We are thinking we should pivot the "FLAG" and the "Predicted Values" column for this, and then we will be able to do all the things aforementioned. But because this table was created using DAX from another table, we are unable to find a way to pivot it. It wont evens how up in the Power Query editor. Any tips or help are welcomed. Sorry again, Im a novice so just learning as I work..
You don't need to pivot. You need a flag dimension table joined to your existing table on flag. The dimension contains just two values for true and false and the cardinality is one-to-many unidirectional. Then on your bar chart, make sure to check show items with no data in the field well.
Starting with : (Using enter data and copy paste from excel)
Adding the following code in the advanced editor under Transforme Data:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WSlTSUQoJCnUFUkbmJsYWSrE60UpJCEFDAzOwUDJCyMTUFCyUgqTKAiHk5ugTDDYOLJKKpMjMBCyUhmy6BVwIps8QLJKOpMjUHCyUgSRkbAJXBdNnAnFoJpIQxFFZqI6KBQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [category = _t, Flag = _t, Value = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"category", type text}, {"Flag", type logical}, {"Value", Int64.Type}}),
#"Pivoted Column" = Table.Pivot(Table.TransformColumnTypes(#"Changed Type", {{"Flag", type text}}, "en-SE"), List.Distinct(Table.TransformColumnTypes(#"Changed Type", {{"Flag", type text}}, "en-SE")[Flag]), "Flag", "Value", List.Sum)
in
#"Pivoted Column"
Gives:
Which is what I assume you are after
Related
my data number came in text form, for example: 4.1M 1.22B 3K
Is there a way to turn them back into numbers like 4100000 for 4.1M
Thanks in advance
If you are using card visualization you have feature to change count for thousands, millions etc. It depend on visualization.
You can follow the following steps:
Select the visual where you want to change this notation
Click on the paint roller in the Visualisations pane
You have already activated labels, expand this part
Here you can choose how to show values, select whatever you want
Keep in mind however that big numbers are hard to read and easy to read wrong. It is worth considering to keep them as 4.1M and 1.22B since it keeps them nice and compact. This is especially the case if you use visualisations that grow over time, where overlap might occur easily if you have written-out numbers.
Also, refer to the following documentation for explanation on the Microsoft website:
https://learn.microsoft.com/en-us/power-bi/visuals/power-bi-visualization-customize-title-background-and-legend
Here is an example of how to clean data in Power Query. It is a bit cumbersome, and personally I would contact whoever is responsible for my data source and ask if I could get data in a better format.
The idea is to first figure out what type of multiplier you want to apply to each row, then clean out the "foreign" symbols in the Sample Value column - this is generated using a nifty trick with Character.FromNumber, see link: https://www.excelguru.ca/blog/2015/11/19/keep-only-numbers-in-power-query/ )after that it is simply to convert the original column to a number (might have to remove #"Replaced Value" step depending on your locale) and create a final column that multiplies the number with the multiplier.
You can paste the below into a blank query to see for yourself:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMtEz9FWK1YlWMtQzMnICs4y9IQLGSrGxAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Sample Values" = _t]),
#"Added Conditional Column" = Table.AddColumn(Source, "Custom", each if Text.Contains([Sample Values], "B") then Number.Power(10,9) else if Text.Contains([Sample Values], "M") then Number.Power(10,6) else if Text.Contains([Sample Values], "K") then Number.Power(10,3) else 1),
CharsToRemove = List.Transform({33..45,47,58..126}, each Character.FromNumber(_)),
#"Added Custom" = Table.AddColumn(#"Added Conditional Column", "Result", each Text.Remove([Sample Values],CharsToRemove)),
#"Replaced Value" = Table.ReplaceValue(#"Added Custom",".",",",Replacer.ReplaceText,{"Result"}),
#"Changed Type" = Table.TransformColumnTypes(#"Replaced Value",{{"Result", type number}}),
#"Added Custom1" = Table.AddColumn(#"Changed Type", "Result Value", each [Result]*[Custom], type number),
#"Removed Other Columns" = Table.SelectColumns(#"Added Custom1",{"Result Value"})
in
#"Removed Other Columns"
I have data from an external source that is downloaded in csv format. This data shows the interactions from several users and doesn't have an id column. The problem I'm having is that I'm not able to use index because multiple entries represent interactions and processes. The interactions would be the group of processes a specific user do and the process represents each actions taken in a specific interaction. Any user could repeat the same interaction at any time of day. The data looks likes this:
User1 has 2 processes but there were 3 interactions. How can I assign an ID for each interaction having into consideration that there might be multiple processes for a single user in the same day. I tried grouping them in Power Query but it groups the overall processes and I'm not able to distinguish the number of interactions. Is it better to do it in Dax?
Edit:
I notice that it is hard to understand what I need but I think this would be a better way to see it:
Process 2 are the steps done in an interaction. Like in the column in yellow I need to add an ID taking in to consideration where an interaction start and where it ends.
I'm not exactly sure I follow what you describe. It looks to me like user1 has 4 interactions--Processes AA, AB, BA, and BB--but you say 3.
Still, I decided to take a shot at providing an answer anyway. I started with a CSV file set up like you show.
Then brought the CSV into Power Query and, just to add a future point of reference so that you could follow the Id assignments better, I added an index column that I called startingIndex.
Then I added a custom column combining the processes that I understand actually define an interaction.
Then I grouped everything by users and Interactions into a column named allData.
Then I added a custom column to copy the column that was created from the earlier grouping, to sort the tables within it, and to add an index to each table within it. This essentially indexed each user's interaction group. (Because all of your interactions occur on the same date(s), the sorting doesn't help much. But I did it to show where you could do it if you included datetime info instead of just a date.)
Then I added a custom column to copy the column that was created earlier to add the interactions index, and to add an Id item within each table within it. I constructed each Id by combining the user, interactions, and interactionIndex for each.
Then I selected the latest column I had created (complexId) and removed all other columns.
Last, I expanded all tables without including the Interactions and Index columns. (The Index column was the index used for the interactions within the groups and no longer needed.) I included the startingIndex column just so you could see where items originally were at the start, in comparison to their final Id.
Given your new example, to create the Interaction ID you show, you only need the first two columns of the table. If not part of the original data, you can easily generate the third column (Process2))
It appears you want to increment the interaction ID whenever the Process changes
Please read the comments in the M code and explore the Applied Steps to better understand the algorithm:
M Code
let
//be sure to change table name in next row to your real table name
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{
{"User", type text},
{"Process", type text},
{"Process2", type text}
}),
//add an index column
idx = Table.AddIndexColumn(#"Changed Type", "Index", 0, 1, Int64.Type),
//Custom column returns the Index if
// the current Index is 0 (first row) or
// there has been no change in user or process comparing current/previous row
// else return null
#"Added Custom" = Table.AddColumn(idx, "Custom",
each if [Index]=0
then 0
else
if [Process] <> idx[Process]{[Index]-1} or [User] <> idx[User]{[Index]-1} then [Index]
else null),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Index"}),
//Fill down the custom column
// now have same number for each interactive group
#"Filled Down" = Table.FillDown(#"Removed Columns",{"Custom"}),
//Group by the "filled down" custom column with no aggregation
#"Grouped Rows" = Table.Group(#"Filled Down", {"Custom"}, {
{"all", each _, type table [User=nullable text, Process=nullable text, Process2=nullable text, Custom=number]}
}),
//add a one-based Index column to the grouped table
#"Added Index" = Table.AddIndexColumn(#"Grouped Rows", "Interaction ID", 1, 1, Int64.Type),
#"Removed Columns1" = Table.RemoveColumns(#"Added Index",{"Custom"}),
//Re-expand the table
#"Expanded all" = Table.ExpandTableColumn(#"Removed Columns1", "all",
{"User", "Process", "Process2"}, {"User", "Process", "Process2"})
in
#"Expanded all"
Source
Results
I have a HTML file that is generated daily. Over the past few years we have added a couple of columns to the HTML table in the file. What I want to do is generate some reports that trend over time based on that HTML, so I want to define a single query for a report, but get a null/default value when the column isn't present in the source.
I have a list of report dates that are available and then I can add copies of the report data to a master report. The data source however fails to load if the column isn't present in the older reports. Essentially I read a date from one HTML file as an input, then modify the fetch URL per row for the source to get the historical data.
Is it possible to generate this report without retrospectively changing the old data and adding the column that is missing? I couldn't see how to do this easily.
I've never done anything like that, but maybe something like this will help.
It checks if a column exists and if it does not then create a new column with a default value.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUUpUitWJVjICspLALGMgK1kpNhYA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [AA = _t, BB = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"AA", Int64.Type}, {"BB", type text}}),
#"Custom1" = if Table.HasColumns(#"Changed Type",{"MissingColumn1"}) = false
then Table.AddColumn(#"Changed Type","MissingColumn1", each "<n/d>")
else #"Changed Type"
,#"Custom2" = if Table.HasColumns(#"Custom1",{"MissingColumn2"}) = false
then Table.AddColumn(#"Custom1","MissingColumn2", each "<n/d>")
else #"Custom1"
,#"Custom3" = if Table.HasColumns(#"Custom2",{"MissingColumn3"}) = false
then Table.AddColumn(#"Custom2","MissingColumn3", each "<n/d>")
else #"Custom2"
in
#"Custom3"
If this is a suitable solution depends on how data are fetched from the source.
I am trying to find a way in Power BI to transform a table into a specific format for a custom visualization. The transformation that I'd like to do is the following image:
Is this possible to do? I've thought of unpivoting or transposing but haven't seemed to figure out how to get past this point.
I do not really understand the use case of this transformation, but you can achieve what you want by copying the source table, remove all columns except 2, and then append all copies.
Let's say we have one table, named Table, like this:
Make 3 copies of the table, by clicking New Source -> Blank Query:
and enter =Table as query text, where Table is the name of our source table. If our queries are named Query1, Query2 and Query3, then click on each of them and remove the extra column and keep A and B in Query1, keep B and C in Query2, and C and D in Query3. Rename the remaining columns to be named the same way in all 3 queries and click Append Queries or Append Queries as New, then select the 3 copes:
This will give you the result you ask for:
Or you could do it in one go, without creating three different queries. Makes your Queries pane a bit nitter
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUXICYmcgdlGKjQUA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [c1 = _t, c2 = _t, c3 = _t, c4 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"c1", type text}, {"c2", type text}, {"c3", type text}, {"c4", type text}}),
_t1 = Table.SelectColumns(#"Changed Type", {"c1", "c2"}),
_t1_r = Table.RenameColumns(_t1, {{"c1", "ca"}, {"c2", "cb"}}),
_t2 = Table.SelectColumns(#"Changed Type", {"c2", "c3"}),
_t2_r = Table.RenameColumns(_t2, {{"c2", "ca"}, {"c3", "cb"}}),
_t3 = Table.SelectColumns(#"Changed Type", {"c3", "c4"}),
_t3_r = Table.RenameColumns(_t3, {{"c3", "ca"}, {"c4", "cb"}}),
_res = Table.Combine({_t1_r, _t2_r, _t3_r})
in
_res
One use case I can think of this transformation will be useful is that, when we have a time series data like this, and we want to compare the values of current and last periods.
So here is a version for the case when the number of columns is variable.
let
Source = ...,
SingleRow = Table.SingleRow(Source),
Values = Record.FieldValues(SingleRow),
Pairs = List.Zip({List.RemoveLastN(Values, 1), List.RemoveFirstN(Values, 1)}),
Result = Table.FromRows(Pairs, {"Last", "Current"})
in
Result
I am implying the "rules" for this transformation are then you want a table where the first column and second column form each "pair" of items in the original set. Given the data loaded into Power Query like this:
I would transpose the data:
Then make two references to the query. In the first one remove the last row, in the second one remove the first row. Then add an index column to both.
(result of the first reference looks like this)
Then just do a merge between the two references and you get the two column outputs show in the example.
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 5 years ago.
Improve this question
I have two different file shows in below table, one is bugtracker and another is bugtracker (2)
Now I want to compare two statuses.
If the status is different, then count it
If all you're really asking for is a True or False comparison as to whether the 'Assigned User Name' and 'Status' of one table's record equals the 'Assigned User Name' and 'Status' of the other table's matching record, then using DAX's if should work.
Assuming you've already matched and merged your "BugTracker" and "BugTracker (2)" table's records in order to get the table you have shown above, and the merged table's name is "BugTrackerMerged", you could just add a column with this DAX command:
Column = if(BugTrackerMerged[Status]=BugTrackerMerged[Status2],TRUE(),FALSE())
Note that I named the second status column 'Status2', instead of 'Status'. Both status columns cannot have the same name.
If you haven't already merged the table's records, you'll need to do that first. I find it easiest to do that with Power Query (Power BI's Edit Queries feature).
(I apologize up front if the following is too detailed. Not knowing your level of Power Query expertise, I figured I'd simplify discussion via step-by-step tutorial. It's more straightforward than it "looks".)
In order to merge the two tables ("BugTracker" and "BugTracker (2)"), you'll need a common keyfield for matching and merging. For this situation, I assume your first record in "BugTracker" should match and merge with the first record of "BugTracker (2)", your second record in "BugTracker" should match and merge with the second record of "BugTracker (2)", and so on. Therefore, just add an index to each table.
For BugTracker, in Power Query select the "BugTracker" query:
Then click the "Add Column" tab, and then "Index Column". (That will add the index to the "BugTracker" table.)
Do the same for "BugTracker (2)".
With common indexes for both "BugTracker" and "BugTracker (2)" you can match and merge the two tables. Click the "Home" tab, then the drop-down arrow beside "Merge Queries", then "Merge Queries as New".
In the window that pops up, make the selections necessary so it looks like this and click "OK":
This creates a new query, likely called "Merge". At this point, I renamed that query to "BugTrackerMerged".
If you select that new query (now named "BugTrackerMerged") and click on "Source", under "Applied Steps"...
You'll see this code in the formula bar:
= Table.NestedJoin(BugTracker,{"Index"},#"BugTracker (2)",{"Index"},"NewColumn",JoinKind.FullOuter)
In that code, change "NewColumn" to "BugTracker (2)" to rename the column that is generated. (You could rename it as a separate step if your prefer, but I thought this approach was "cleaner".
Then click the button, to the right of the "BugTracker (2)" column's title...
...to expand the tables in the column. You'll see a pop-up window like this:
Leaving the settings like shown here will expand (bring in) all the columns from the secondary table of the earlier merge. (That secondary table was "BugTracker (2)".) Using the original column name as prefix will help you keep straight which "Status" and "Assigned User Name" info comes from which table.
At this point, you have the merged info. You could go one step further here and do the True/False comparison here too as well, if you like. To do that, just add a new custom column with some code: click the "Add Column" tab, and the "Custom Column" button:
Then, in the pop-up window, add this code:
if [Status]&[Assigned User Name]=[#"BugTracker (2).Status"]&[#"BugTracker (2).Assigned User Name"] then "True" else "False"
Like this:
You'll get a table like this:
Your data has a lot of "Trues" up front. You can easily see that there are also "Falses" though, by using the column's filter button.
Here's my Power Query (M) code for my three queries:
BugTracker:
let
Source = Excel.Workbook(File.Contents("C:\Users\MARC_000\Desktop\sample\Rowdata Programming 15 July 2017 (2).xlsx"), null, true),
BugTracker_Sheet = Source{[Item="BugTracker",Kind="Sheet"]}[Data],
#"Changed Type" = Table.TransformColumnTypes(BugTracker_Sheet,{{"Column1", type text}, {"Column2", type text}}),
#"Promoted Headers" = Table.PromoteHeaders(#"Changed Type", [PromoteAllScalars=true]),
#"Added Index" = Table.AddIndexColumn(#"Promoted Headers", "Index", 0, 1)
in
#"Added Index"
BugTracker (2):
let
Source = Excel.Workbook(File.Contents("C:\Users\MARC_000\Desktop\sample\Rowdata Programming 18 July 2017.xlsx"), null, true),
BugTracker_Sheet = Source{[Item="BugTracker",Kind="Sheet"]}[Data],
#"Changed Type" = Table.TransformColumnTypes(BugTracker_Sheet,{{"Column1", type text}, {"Column2", type text}}),
#"Promoted Headers" = Table.PromoteHeaders(#"Changed Type", [PromoteAllScalars=true]),
#"Added Index" = Table.AddIndexColumn(#"Promoted Headers", "Index", 0, 1)
in
#"Added Index"
BugTrackerMerged:
let
Source = Table.NestedJoin(BugTracker,{"Index"},#"BugTracker (2)",{"Index"},"BugTracker (2)",JoinKind.FullOuter),
#"Expanded BugTracker (2)" = Table.ExpandTableColumn(Source, "BugTracker (2)", {"Status", "Assigned User Name", "Index"}, {"BugTracker (2).Status", "BugTracker (2).Assigned User Name", "BugTracker (2).Index"}),
#"Added Custom" = Table.AddColumn(#"Expanded BugTracker (2)", "Custom", each if [Status]&[Assigned User Name]=[#"BugTracker (2).Status"]&[#"BugTracker (2).Assigned User Name"] then "True" else "False")
in
#"Added Custom"