I want to profile every single data table I have in my Power BI report. By data profile I mean something like this:
Are there ways to make a data profile view in Power BI? DAX measure or calculated columns?
Alternatively, you can also recommend other data quality tools that can handle such tasks since I find it a bit difficult to achieve this result in Power BI.
Now I feel dumb after writing a manual query that did what it turns out Table.Profile does in one shot. However I will mention you can automatically get a profile for every table in your data set by using the #shared reference and filtering down to the tables:
let
Source = #shared,
#"Converted to Table" = Record.ToTable(Source),
#"Added Custom" = Table.AddColumn(#"Converted to Table", "TableCheck", each Type.Is(Value.Type([Value]), type table)),
#"Filtered Rows" = Table.SelectRows(#"Added Custom", each ([TableCheck] = true)),
#"Filtered Rows1" = Table.SelectRows(#"Filtered Rows", each [Name] <> "NAME_OF_THIS_QUERY"),
#"Added Custom1" = Table.AddColumn(#"Filtered Rows1", "Profile", each Table.Profile([Value])),
#"Expanded Profile" = Table.ExpandTableColumn(#"Added Custom1", "Profile", {"Column", "Min", "Max", "Average", "StandardDeviation", "Count", "NullCount", "DistinctCount"}, {"Profile.Column", "Profile.Min", "Profile.Max", "Profile.Average", "Profile.StandardDeviation", "Profile.Count", "Profile.NullCount", "Profile.DistinctCount"})
in
#"Expanded Profile"
And replace "NAME_OF_THIS_QUERY" with whatever you name the query so it doesn't try to profile itself.
Power BI has a built-in data profiler
Open Power BI and refer to the menu ribbon
Click Home
Click Edit Queries
Click View
Select Column Profile to view stats about your data
In the query editor, you can use the Table.Profile function on any table.
You can do multiple ones simultaneously like this:
= Table.Combine({Table.Profile(Table1),Table.Profile(Table2)})
Edit:
To see the profile, create a new Blank Query and define it as = Table.Profile(Table1). If you open the Advanced Editor, the M code looks like this:
let
Source = Table.Profile(Table1)
in
Source
Related
I need to create something like this using DAX. Preferably highlighted columns as two columns of a table. I have managed to set the preview of the requirement But I'm unable to proceed with it. Hope someone can help me with this.
I have tried with DAX queries, but it won't give me what I need to be done.
This feels more like a Power Query to me. Try pasting this into a blank query in the Advanced Editor:
let
StartingValue = 8,
Deduction = 0.10,
StartDate = #date(2023,2,14),
Times=6,
#"Multiplier" = 1 - Deduction,
Source = Table.FromList({1..Times}, Splitter.SplitByNothing(), type table [Index=Int64.Type], null, ExtraValues.Error),
#"Add Amount" = Table.AddColumn(Source, "Amount", each StartingValue * Number.Power(#"Multiplier",[Index]-1), type number),
#"Add Date" = Table.AddColumn(#"Add Amount", "Date", each Date.AddMonths(StartDate,[Index]-1), type date)
in
#"Add Date"
You can change the 4 input values into Parameters and make this a function in Power Query to easily create new tables.
I have added SQL data source. Then, instead of selecting the table I have typed the following query:
SELECT ID, NAME, AGE
FROM STUDENTS
When I right click on the step then I see that the native query option is disabled. Why does adding simple SQL query as Power BI data source prevent native query folding? Is native query only supported if I select the table via UI and then select the transformations?
You can enable query folding with a native query. There is an excellent article by Chris Webb where he describes all the details.
https://blog.crossjoin.co.uk/2021/02/21/query-folding-on-sql-queries-in-power-query-using-value-nativequery-and-enablefoldingtrue/
EDIT: CODE ADDED
let
Source = Sql.Databases("localhost"),
AdventureWorksDW2017 = Source
{[Name = "AdventureWorksDW2017"]}
[Data],
RunSQL = Value.NativeQuery(
AdventureWorksDW2017,
"SELECT EnglishDayNameOfWeek FROM DimDate",
null,
[EnableFolding = true]
),
#"Filtered Rows" = Table.SelectRows(
RunSQL,
each (
[EnglishDayNameOfWeek] = "Friday"
)
)
in
#"Filtered Rows"
How to set up filters for date before getting full data. Right now pbi export all data from day one. Is more than 1.7mil. rows. I reach limit after 1mil and way to use date filter inside pbi is not a option.
So are posible way setdate range parameter (example last 3 month) before export starts?
In adv editor source code:
let
Source = GoogleAnalytics.Accounts(),
#"1*******1" = Source{[Id="1*******1"]}[Data],
#"UA-1*******1-1" = #"1*******1"{[Id="UA-1*******1-1"]}[Data],
#"1*******6" = #"UA-1*******1-1"{[Id="1*******6"]}[Data],
#"Added Items" = Cube.Transform(#"1*******6",
{
{Cube.AddAndExpandDimensionColumn, "ga:eventAction", {"ga:eventAction"}, {"Event Action"}},
{Cube.AddAndExpandDimensionColumn, "ga:eventLabel", {"ga:eventLabel"}, {"Event Label"}},
/*{Cube.AddAndExpandDimensionColumn, "ga:date", {"ga:date"}, {"Date"}},*/
{Cube.AddMeasureColumn, "Unique Events", "ga:uniqueEvents"}
})
in
#"Added Items"
#"1*******1" = Source{[Id="1*******1"]}[Data],
#"UA-1*******1-1" = #"1*******1"{[Id="UA-1*******1-1"]}[Data],
#"1*******6" = #"UA-1*******1-1"{[Id="1*******6"]}[Data],
My guess, after source location we need somehow set date range parameters.
I know that it is an old question and supposed that you solve it.
I think it may be useful for those who google this question.
After step #"Added Items" replace text (in #"Added Items")
(be aware first in SelectRows( should be the name of the previous step)
with next
#"combinedData" = Table.Combine({
Table.SelectRows(#"Added Items", each Text.Contains([Month of Year], "2020"))
Table.SelectRows(#"Added Items", each Text.Contains([Month of Year], "2019"))
})
in
combinedData
I'm trying to model a data source in power bi, but I'm not getting it.
Could you help me with alternatives so I can create a new column? The data source is in excel and brings the data with subtotal by types (XPTO, XPT, etc). I want to put these types as corresponding values in the new column for your items. I tried via power query and dax, but I could not.
Original Source:
Modifications Needed
Source File
This assumes that you can identify the Subtotal Rows by checking the position of the first space in the first column:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Added RowType" = Table.AddColumn(Source, "RowType", each Text.PositionOf([#"Centro financ./item orçamento"]," "), Int64.Type),
#"Added Type" = Table.AddColumn(#"Added RowType", "Type", each if [RowType] = 4 then Text.BetweenDelimiters([#"Centro financ./item orçamento"], " ", " ") else null, type text),
#"Filled Down" = Table.FillDown(#"Added Type",{"Type"}),
#"Filtered Rows" = Table.SelectRows(#"Filled Down", each ([RowType] = 8)),
#"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"RowType"}),
#"Reordered Columns" = Table.ReorderColumns(#"Removed Columns",List.Combine({{"Type"}, Table.ColumnNames(Source)}))
in
#"Reordered Columns"
Another solution is to - Include another data source in Power BI, which will look something like this.
Item Type
615x92120 Mat1 XPTA
615x92121 Mat2 XPTA
615x92122 Mat3 XPTU
615x92123 Mat4 XPTU
And then do a Join between your existing table and this table to bring out the Type in your existing table. Once you have done that, you should be able to filter out to blanks or null which will be your delete lines.
Note :- This only works, if you know all the items and corresponding types in advance.
let
Source = Sql.Databases("xx.x.x.x"),
zzz_LIVE = Source{[Name="NAVzzz_LIVE"]}[Data],
#"COMPANY1$G_L Entry" = NAVzzzLIVE{[Schema="dbo",Item="COMPANY1$G_L Entry"]}[Data],
Pulling data from several entities (companies) in Microsoft Dynamics Navision with above code (Company1 as an example).
Trying to add a custom column with an unique identifier based on the name of the company.
G_L Account number G_L Account name Amount Company
10010 Revenue 100 Company1
22000 Rent 50 Company1
Is it possible to achieve this by M code?
Many thanks.
How about something like this?
let
Source = Sql.Databases("xx.x.x.x"),
zzz_LIVE = Source{[Name="NAVzzz_LIVE"]}[Data],
Name1 = "Company1",
Name2 = "Company2",
[...]
#"COMPANY1" = NAVzzzLIVE{[Schema="dbo",Item=Name1&"$G_L Entry"]}[Data],
#"Added Custom1" = Table.AddColumn(#"COMPANY1", "Company", each Name1),
#"COMPANY2" = NAVzzzLIVE{[Schema="dbo",Item=Name2&"$G_L Entry"]}[Data],
#"Added Custom2" = Table.AddColumn(#"COMPANY2", "Company", each Name2),
[...]
Table.Combine(#"Added Custom1",#"Added Custom2",[...])
That way you
I am not able to test this, but I think it might work for you:
let
Source = Sql.Databases("xx.x.x.x"),
zzz_LIVE = Source{[Name="NAVzzz_LIVE"]}[Data],
#"COMPANY1$G_L EntrySrc" = NAVzzzLIVE{[Schema="dbo",Item="COMPANY1$G_L Entry"]},
#"COMPANY1$G_L Entry" = #"COMPANY1$G_L EntrySrc"[Data],
#"Added Custom" = Table.AddColumn(#"COMPANY1$G_L Entry", "Company",
each Record.Field(#"COMPANY1$G_L EntrySrc", "Item"))
in
#"Added Custom"
I split the Navigation step (#"COMPANY1$G_L Entry") in two so that I could reference its record. Otherwise, the [Data] gets in the way.
If I'm correct, it will return COMPANY1$G_L Entry into your new column. You could change Record.Field(#"COMPANY1$G_L EntrySrc", "Item") to Text.Before(Record.Field(#"COMPANY1$G_L EntrySrc", "Item"), "$G_L") to only return COMPANY1.