I am running a query to update a historical data table within PowerBI. This update query calls the Adobe Analytics API, and pulls data based on a date range I specify.
As I want this to run as fast as possible, I would only like to query the data that I don't already have in my historical data table. I am okay at putting an end date for this query based off Date.Time.LocalNow().
#date(Date.Year(DateTime.LocalNow()), Date.Month(DateTime.LocalNow()), Date.Day(DateTime.LocalNow())-1)
However, where I am stuck is putting a start date. Ideally I would like to take the MaxDate from a table within my data model, and use this value in the query editor as my start date.
How would I do this?
UPDATE 1 - Full code Below
let
Source = AdobeAnalytics.Cubes()
myreportsuiteid = Source{[Id="myreportsuiteid"]}[Data],
#"Added Items" = Cube.Transform(myreportsuiteid,
{
{Cube.AddAndExpandDimensionColumn, "DateGranularity", {"year", "month", "day"}, {"Date Granularity.Level 1: Year", "Date Granularity.Level 2: Month", "Date Granularity.Level 3: Day"}},
{Cube.AddAndExpandDimensionColumn, "lasttouchchannel", {"lasttouchchannel"}, {"Last Touch Marketing Channel"}},
{Cube.AddMeasureColumn, "Unique Visitors", "uniquevisitors"},
{Cube.AddMeasureColumn, "Visits", "visits"},
{Cube.ApplyParameter, "DateRange", {#date(Date.Year(List.Max(Union[Date])), Date.Month(List.Max(Union[Date])), Date.Day(List.Max(Union[Date]))+1), #date(Date.Year(DateTime.LocalNow()), Date.Month(DateTime.LocalNow()), Date.Day(DateTime.LocalNow())-1)}},
{Cube.ApplyParameter, "Segment", {{"s300000554_5ae201be22fa9950dcdbcd92"}}}
})
in
#"Added Items"
UPDATE 2
WORKING CODE
Daily US is a table created thorough the Query Editor
let
Source = AdobeAnalytics.Cubes(),
todaysDate = Date.AddDays(DateTime.Date(DateTime.LocalNow()),-1),
maxDate = Date.AddDays(List.Max(#"Daily US"[Date]),1),
myreportsuiteid = Source{[Id="myreportsuiteid "]}[Data],
ERROR CODE
Union US is a table created outside of the Query Editor via Union US = DISTINCT(UNION('Seeker US','Daily US'))
let
Source = AdobeAnalytics.Cubes(),
todaysDate = Date.AddDays(DateTime.Date(DateTime.LocalNow()),-1),
maxDate = Date.AddDays(List.Max(#"Union US"[Date]),1),
myreportsuiteid = Source{[Id="myreportsuiteid "]}[Data],
If the column you want to reference for the MaxDate is named Union[Date], then you should be able to use List.Max to get the maximum value from that column.
Try this:
let
Source = AdobeAnalytics.Cubes(),
MaxDate = List.Max(Union[Date]),
TodaysDate = DateTime.Date(DateTime.LocalNow()),
myreportsuiteid = Source{[Id="myreportsuiteid"]}[Data],
#"Added Items" = Cube.Transform(myreportsuiteid,
{
{Cube.AddAndExpandDimensionColumn, "DateGranularity", {"year", "month", "day"}, {"Date Granularity.Level 1: Year", "Date Granularity.Level 2: Month", "Date Granularity.Level 3: Day"}},
{Cube.AddAndExpandDimensionColumn, "lasttouchchannel", {"lasttouchchannel"}, {"Last Touch Marketing Channel"}},
{Cube.AddMeasureColumn, "Unique Visitors", "uniquevisitors"},
{Cube.AddMeasureColumn, "Visits", "visits"},
{Cube.ApplyParameter, "DateRange", {MaxDate, TodaysDate}},
{Cube.ApplyParameter, "Segment", {{"s300000554_5ae201be22fa9950dcdbcd92"}}}
})
in
#"Added Items"
Related
I need to create something like this using DAX. Preferably highlighted columns as two columns of a table. I have managed to set the preview of the requirement But I'm unable to proceed with it. Hope someone can help me with this.
I have tried with DAX queries, but it won't give me what I need to be done.
This feels more like a Power Query to me. Try pasting this into a blank query in the Advanced Editor:
let
StartingValue = 8,
Deduction = 0.10,
StartDate = #date(2023,2,14),
Times=6,
#"Multiplier" = 1 - Deduction,
Source = Table.FromList({1..Times}, Splitter.SplitByNothing(), type table [Index=Int64.Type], null, ExtraValues.Error),
#"Add Amount" = Table.AddColumn(Source, "Amount", each StartingValue * Number.Power(#"Multiplier",[Index]-1), type number),
#"Add Date" = Table.AddColumn(#"Add Amount", "Date", each Date.AddMonths(StartDate,[Index]-1), type date)
in
#"Add Date"
You can change the 4 input values into Parameters and make this a function in Power Query to easily create new tables.
How to set up filters for date before getting full data. Right now pbi export all data from day one. Is more than 1.7mil. rows. I reach limit after 1mil and way to use date filter inside pbi is not a option.
So are posible way setdate range parameter (example last 3 month) before export starts?
In adv editor source code:
let
Source = GoogleAnalytics.Accounts(),
#"1*******1" = Source{[Id="1*******1"]}[Data],
#"UA-1*******1-1" = #"1*******1"{[Id="UA-1*******1-1"]}[Data],
#"1*******6" = #"UA-1*******1-1"{[Id="1*******6"]}[Data],
#"Added Items" = Cube.Transform(#"1*******6",
{
{Cube.AddAndExpandDimensionColumn, "ga:eventAction", {"ga:eventAction"}, {"Event Action"}},
{Cube.AddAndExpandDimensionColumn, "ga:eventLabel", {"ga:eventLabel"}, {"Event Label"}},
/*{Cube.AddAndExpandDimensionColumn, "ga:date", {"ga:date"}, {"Date"}},*/
{Cube.AddMeasureColumn, "Unique Events", "ga:uniqueEvents"}
})
in
#"Added Items"
#"1*******1" = Source{[Id="1*******1"]}[Data],
#"UA-1*******1-1" = #"1*******1"{[Id="UA-1*******1-1"]}[Data],
#"1*******6" = #"UA-1*******1-1"{[Id="1*******6"]}[Data],
My guess, after source location we need somehow set date range parameters.
I know that it is an old question and supposed that you solve it.
I think it may be useful for those who google this question.
After step #"Added Items" replace text (in #"Added Items")
(be aware first in SelectRows( should be the name of the previous step)
with next
#"combinedData" = Table.Combine({
Table.SelectRows(#"Added Items", each Text.Contains([Month of Year], "2020"))
Table.SelectRows(#"Added Items", each Text.Contains([Month of Year], "2019"))
})
in
combinedData
let
Source = Sql.Databases("xx.x.x.x"),
zzz_LIVE = Source{[Name="NAVzzz_LIVE"]}[Data],
#"COMPANY1$G_L Entry" = NAVzzzLIVE{[Schema="dbo",Item="COMPANY1$G_L Entry"]}[Data],
Pulling data from several entities (companies) in Microsoft Dynamics Navision with above code (Company1 as an example).
Trying to add a custom column with an unique identifier based on the name of the company.
G_L Account number G_L Account name Amount Company
10010 Revenue 100 Company1
22000 Rent 50 Company1
Is it possible to achieve this by M code?
Many thanks.
How about something like this?
let
Source = Sql.Databases("xx.x.x.x"),
zzz_LIVE = Source{[Name="NAVzzz_LIVE"]}[Data],
Name1 = "Company1",
Name2 = "Company2",
[...]
#"COMPANY1" = NAVzzzLIVE{[Schema="dbo",Item=Name1&"$G_L Entry"]}[Data],
#"Added Custom1" = Table.AddColumn(#"COMPANY1", "Company", each Name1),
#"COMPANY2" = NAVzzzLIVE{[Schema="dbo",Item=Name2&"$G_L Entry"]}[Data],
#"Added Custom2" = Table.AddColumn(#"COMPANY2", "Company", each Name2),
[...]
Table.Combine(#"Added Custom1",#"Added Custom2",[...])
That way you
I am not able to test this, but I think it might work for you:
let
Source = Sql.Databases("xx.x.x.x"),
zzz_LIVE = Source{[Name="NAVzzz_LIVE"]}[Data],
#"COMPANY1$G_L EntrySrc" = NAVzzzLIVE{[Schema="dbo",Item="COMPANY1$G_L Entry"]},
#"COMPANY1$G_L Entry" = #"COMPANY1$G_L EntrySrc"[Data],
#"Added Custom" = Table.AddColumn(#"COMPANY1$G_L Entry", "Company",
each Record.Field(#"COMPANY1$G_L EntrySrc", "Item"))
in
#"Added Custom"
I split the Navigation step (#"COMPANY1$G_L Entry") in two so that I could reference its record. Otherwise, the [Data] gets in the way.
If I'm correct, it will return COMPANY1$G_L Entry into your new column. You could change Record.Field(#"COMPANY1$G_L EntrySrc", "Item") to Text.Before(Record.Field(#"COMPANY1$G_L EntrySrc", "Item"), "$G_L") to only return COMPANY1.
I want to profile every single data table I have in my Power BI report. By data profile I mean something like this:
Are there ways to make a data profile view in Power BI? DAX measure or calculated columns?
Alternatively, you can also recommend other data quality tools that can handle such tasks since I find it a bit difficult to achieve this result in Power BI.
Now I feel dumb after writing a manual query that did what it turns out Table.Profile does in one shot. However I will mention you can automatically get a profile for every table in your data set by using the #shared reference and filtering down to the tables:
let
Source = #shared,
#"Converted to Table" = Record.ToTable(Source),
#"Added Custom" = Table.AddColumn(#"Converted to Table", "TableCheck", each Type.Is(Value.Type([Value]), type table)),
#"Filtered Rows" = Table.SelectRows(#"Added Custom", each ([TableCheck] = true)),
#"Filtered Rows1" = Table.SelectRows(#"Filtered Rows", each [Name] <> "NAME_OF_THIS_QUERY"),
#"Added Custom1" = Table.AddColumn(#"Filtered Rows1", "Profile", each Table.Profile([Value])),
#"Expanded Profile" = Table.ExpandTableColumn(#"Added Custom1", "Profile", {"Column", "Min", "Max", "Average", "StandardDeviation", "Count", "NullCount", "DistinctCount"}, {"Profile.Column", "Profile.Min", "Profile.Max", "Profile.Average", "Profile.StandardDeviation", "Profile.Count", "Profile.NullCount", "Profile.DistinctCount"})
in
#"Expanded Profile"
And replace "NAME_OF_THIS_QUERY" with whatever you name the query so it doesn't try to profile itself.
Power BI has a built-in data profiler
Open Power BI and refer to the menu ribbon
Click Home
Click Edit Queries
Click View
Select Column Profile to view stats about your data
In the query editor, you can use the Table.Profile function on any table.
You can do multiple ones simultaneously like this:
= Table.Combine({Table.Profile(Table1),Table.Profile(Table2)})
Edit:
To see the profile, create a new Blank Query and define it as = Table.Profile(Table1). If you open the Advanced Editor, the M code looks like this:
let
Source = Table.Profile(Table1)
in
Source
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 5 years ago.
Improve this question
I have two different file shows in below table, one is bugtracker and another is bugtracker (2)
Now I want to compare two statuses.
If the status is different, then count it
If all you're really asking for is a True or False comparison as to whether the 'Assigned User Name' and 'Status' of one table's record equals the 'Assigned User Name' and 'Status' of the other table's matching record, then using DAX's if should work.
Assuming you've already matched and merged your "BugTracker" and "BugTracker (2)" table's records in order to get the table you have shown above, and the merged table's name is "BugTrackerMerged", you could just add a column with this DAX command:
Column = if(BugTrackerMerged[Status]=BugTrackerMerged[Status2],TRUE(),FALSE())
Note that I named the second status column 'Status2', instead of 'Status'. Both status columns cannot have the same name.
If you haven't already merged the table's records, you'll need to do that first. I find it easiest to do that with Power Query (Power BI's Edit Queries feature).
(I apologize up front if the following is too detailed. Not knowing your level of Power Query expertise, I figured I'd simplify discussion via step-by-step tutorial. It's more straightforward than it "looks".)
In order to merge the two tables ("BugTracker" and "BugTracker (2)"), you'll need a common keyfield for matching and merging. For this situation, I assume your first record in "BugTracker" should match and merge with the first record of "BugTracker (2)", your second record in "BugTracker" should match and merge with the second record of "BugTracker (2)", and so on. Therefore, just add an index to each table.
For BugTracker, in Power Query select the "BugTracker" query:
Then click the "Add Column" tab, and then "Index Column". (That will add the index to the "BugTracker" table.)
Do the same for "BugTracker (2)".
With common indexes for both "BugTracker" and "BugTracker (2)" you can match and merge the two tables. Click the "Home" tab, then the drop-down arrow beside "Merge Queries", then "Merge Queries as New".
In the window that pops up, make the selections necessary so it looks like this and click "OK":
This creates a new query, likely called "Merge". At this point, I renamed that query to "BugTrackerMerged".
If you select that new query (now named "BugTrackerMerged") and click on "Source", under "Applied Steps"...
You'll see this code in the formula bar:
= Table.NestedJoin(BugTracker,{"Index"},#"BugTracker (2)",{"Index"},"NewColumn",JoinKind.FullOuter)
In that code, change "NewColumn" to "BugTracker (2)" to rename the column that is generated. (You could rename it as a separate step if your prefer, but I thought this approach was "cleaner".
Then click the button, to the right of the "BugTracker (2)" column's title...
...to expand the tables in the column. You'll see a pop-up window like this:
Leaving the settings like shown here will expand (bring in) all the columns from the secondary table of the earlier merge. (That secondary table was "BugTracker (2)".) Using the original column name as prefix will help you keep straight which "Status" and "Assigned User Name" info comes from which table.
At this point, you have the merged info. You could go one step further here and do the True/False comparison here too as well, if you like. To do that, just add a new custom column with some code: click the "Add Column" tab, and the "Custom Column" button:
Then, in the pop-up window, add this code:
if [Status]&[Assigned User Name]=[#"BugTracker (2).Status"]&[#"BugTracker (2).Assigned User Name"] then "True" else "False"
Like this:
You'll get a table like this:
Your data has a lot of "Trues" up front. You can easily see that there are also "Falses" though, by using the column's filter button.
Here's my Power Query (M) code for my three queries:
BugTracker:
let
Source = Excel.Workbook(File.Contents("C:\Users\MARC_000\Desktop\sample\Rowdata Programming 15 July 2017 (2).xlsx"), null, true),
BugTracker_Sheet = Source{[Item="BugTracker",Kind="Sheet"]}[Data],
#"Changed Type" = Table.TransformColumnTypes(BugTracker_Sheet,{{"Column1", type text}, {"Column2", type text}}),
#"Promoted Headers" = Table.PromoteHeaders(#"Changed Type", [PromoteAllScalars=true]),
#"Added Index" = Table.AddIndexColumn(#"Promoted Headers", "Index", 0, 1)
in
#"Added Index"
BugTracker (2):
let
Source = Excel.Workbook(File.Contents("C:\Users\MARC_000\Desktop\sample\Rowdata Programming 18 July 2017.xlsx"), null, true),
BugTracker_Sheet = Source{[Item="BugTracker",Kind="Sheet"]}[Data],
#"Changed Type" = Table.TransformColumnTypes(BugTracker_Sheet,{{"Column1", type text}, {"Column2", type text}}),
#"Promoted Headers" = Table.PromoteHeaders(#"Changed Type", [PromoteAllScalars=true]),
#"Added Index" = Table.AddIndexColumn(#"Promoted Headers", "Index", 0, 1)
in
#"Added Index"
BugTrackerMerged:
let
Source = Table.NestedJoin(BugTracker,{"Index"},#"BugTracker (2)",{"Index"},"BugTracker (2)",JoinKind.FullOuter),
#"Expanded BugTracker (2)" = Table.ExpandTableColumn(Source, "BugTracker (2)", {"Status", "Assigned User Name", "Index"}, {"BugTracker (2).Status", "BugTracker (2).Assigned User Name", "BugTracker (2).Index"}),
#"Added Custom" = Table.AddColumn(#"Expanded BugTracker (2)", "Custom", each if [Status]&[Assigned User Name]=[#"BugTracker (2).Status"]&[#"BugTracker (2).Assigned User Name"] then "True" else "False")
in
#"Added Custom"