I am working with a bot, and I need to sort the event by a datetime column, when I import the column from my database I have this format:
2017-10-19T14:26:57.2349278Z
after importing in power BI and changing the data type to date time I get this:
10/19/2017 02:26:57 PM
all the trailing milliseconds are truncated, but I need them to sort correctly the events, because some events occurs in the same second.
Any body any idea?
Thanks!
Starting with a sample table of dates in your format, when loaded into Power BI, this is the result.
Click on the Navigation step and then click on Add Column -> Custom Column. A prompt will pop up asking if you want to insert a step, click Insert.
In the prompt, enter the following formula.
Text.Start([#"Timestamp"], 4) &
Text.Middle([#"Timestamp"], 5, 2) &
Text.Middle([#"Timestamp"], 8, 2) &
Text.Middle([#"Timestamp"], 11, 2) &
Text.Middle([#"Timestamp"], 14, 2) &
Text.Middle([#"Timestamp"], 17, 2) &
Text.Middle([#"Timestamp"], 20, 7)
This gives you a column that can be used to sort the items in the table by a precise datetime. Note that the custom column formula requires the raw date string to be in the exact same format as in your question.
Related
I am working on a Leave Tracker in MS LISTS, where in the Calendar view I want the Proxy name to be displayed.
I have searched more than 100 Links and I couldn't the answer.
Do we have any workaround for it?
Thanks for the answer in advance.
According to my research, the “Person Or Group” column is not supported in the SharePoint Calculated column
SharePoint Calculated column supports the following types of columns:
Single line of text
Choice (menu to choose from) | Single Selection
Number (1, 1.0, 100)
Currency ($, ¥, €)
Date and Time
Yes/No (checkbox)
Calculated (calculation based on other columns)
Task Outcome
External Data
Content-Type Columns
We recommend that you can create a Single line of text column to fill in the Proxy Name.
Then, you can create a Calculated column which concatenates the Title, and Proxy Name(Single line of text) columns,
Modify the Calendar view and change the "Month View Title" column to your new Calculated column.
I have a query in Power BI that takes two parameter: Start Date and End Date.
Whenever I pass these Dates it return a table of Date that contain few columns created according to this range of date such as Date, QuarterofYear, Year, MonthName......etc.
Can we create a mapping data flow in ADF that takes two parameter as input and return a calculated table according to provided dates?
Is there any function that return the range of dates?
For your request: "I want that I pass two date Start Date and End Date in ADF Mapping Data Flow , and Data flow will Create a column such as "Date" that contain that number of Date rows. Is there any function for this? Exam. Start Date=20-01-2019, End Date=20-01-2020 Then Date Column Values should be: 20-01-2019 21-01-2019 ......... ......... 20-02-2020", according the Data Factory documents and my experience, the answer is no, we can't achieve it in Data Flow.
There is a solution to this, but it is a bit tricky.
TL;DR
The general data flow looks like this:
We need a dummy source with exactly one row which contains whatever.
Then we derive a column where we use the mapLoop() expression to create an array of all the dates we want to get rows for.
Finally, we need to flatten the array column which will result in one row per array entry and thus one row per date.
Walkthrough
Source dummy
Each dataflow needs a source and we need exactly one row to make our dataflow work. To achieve this I've created a dataset called empty of type CSV in my data lake which has this content:
empty
""
This is our source definition:
And its result looks like this:
Derived column days
This is where the magic happens!
We create a new column dates which is an array of all the dates we want to have in our date table:
In this scenario we want a date table starting on 2019-01-01 and reaching one year into the future. The full expression looks like this:
mapLoop(
addDays(currentDate(), 365) - toDate(2019-01-01),
addDays(toDate(2019-01-01), #index)
)
This is what happens here:
the mapLoop() function builds an array of elements. You specify the number of elements you want to have and the lambda expression to calculate each of the elements. For example, mapIndex([1, 2, 3, 4], #item + 2 + #index) results in [4, 6, 8, 10]
addDays(currentDate(), 365) - toDate('2019-01-01') is the number of days between our start (2019-01-01) and end date (1 year in the future from now) and thus the number of dates we want to have in our resulting array.
addDays(toDate(2019-01-01), #index) calculates each array item by adding #index days to our start date. This is executed for the number of days we've calculated before and #index is the array position. Thus, the first element of the array will be 2019-01-01 + 1, the second 2019-01-01 + 2 and so on.
Our stream now has these columns:
Flatten
Finally, you need a flatten transformation which will expand each item in your array to its dedicated row. We can also dismiss the useless empty column in this step:
And this finally results in what we wanted to achieve:
References
Data transformation expressions in mapping data flow
I have a table containing timestamps and Error codes from machines.
The machines will sometimes repeat the same error several times in a row but i only want to count these as one error. Thus I'm looking for a way to calculate if these errors is reoccurring and filter out these errors with some kind of filter.
I'm using DirectQuery so using EARLIER() to get the last error does not seem to work.
How should i filter these reoccuring errors?
If you want to do this in the database, Azure SQL Database supports LAG function, so the query for loading the data to Power BI could be something like this:
declare #t table([Time] time, [Error] int)
insert into #t([Time], [Error]) values
('11:01', 0),
('12:12', 0),
('13:31', 4),
('14:50', 0),
('15:10', 4),
('15:20', 4),
('15:30', 4),
('15:40', 4),
('17:01', 1),
('18:09', 1),
('19:41', 0)
select
t.[Time]
, t.[Error]
, IIF(t.[Error] <> 0 and LAG(t.[Error], 1) OVER(ORDER BY t.[Time]) = t.[Error], 1, 0) as Reoccuring
from #t t
order by t.[Time]
Please note, that the example doesn't show partitioning the data, e.g. by machine or something, because your sample data doesn't include that. If you need to do it, you must add PARTITION BY clause to the LAG function. If you update your question with exact database schema, I will update my answer too.
As Andrey Nikolov assumed i needed to use PARTITION BY clause using the serial numbers for the machines.
SELECT TOP 100 PERCENT *,
(CASE WHEN error = 0 OR error = LAG(error, 1, 0) OVER (PARTITION BY serial_nr ORDER BY event_time DESC)
THEN 0
ELSE 1
END) AS error_is_new
FROM MyTable
I added a new column in my table containing whether an error is new.
I used error_is_new to only show the errors that were new.
Trying to create a new query from the existing "Master" Query using below formula:
let
Source = Table.SelectColumns('Original Source Name',{'Column Name','Column Name2'})
in
Source
which works fine, however I am looking to see if there is any other formula which would do the same but in a way that it will create the new query with a range of columns , for example Column 30- 67 ( in this case when the original Excel file is updated, inserting a column in this range it would automatically update in the PBI too when refreshed)
Here's one possible way. If you start with this table, named Table1:
You can reference it in a new query like this:
let
Source = Table.SelectColumns(Table1, List.Range(Table.ColumnNames(Table1), 2, 3))
in
Source
...to get this:
The formula selects a range of columns from the table starting at the column at index position 2, and spanning 3 columns. (The index starts with 0.) For columns 30-67, you would change the 2 to 31 and the 3 to 37. You would change Table1 to your Original Source Name as well.
See these links for more info on List.Range and Table.ColumnNames.
I have this custom date that I created as a measure:
Start Date = DATE(YEAR(MAX(Loss[dte_month_end]))-1,12,31)
So this part looks fine in PowerBI and seems to be the right format.
So now I created a new column where I'm going through my data to check whether a record is equal to my "Start Date" as defined above.
IsStart = IF(Loss[dte_month_end]=[Start Date], TRUE, FALSE)
but the weird thing is that all records are evaluated to false.
I know this is actually not the case in my actual data, and I could find actual records with dte_month_end = 12/31/2017 as shown above.
Can someone help me understand why the IF statement would not be able to evaluate this correctly? I initially thought that this may be a case of the DATETIME format being inconsistent - but I purposefully changed both formats to be the same to no avail.
Thanks.
Edit1----------- FYI:
This is the format that my dte_month_end field has:
Edit2 --
I tried changing the dte_month_end format to Date instead of DateTime, and it still doesn't seem to work:
This is happening because you are using a measure inside of a calculated column. When you do this, the filter context for the measure is the row context in the table.
To fix this, you need to modify the filter context for your measure. For example:
Start Date = DATE(YEAR(CALCULATE(MAX(Loss[dte_month_end]), ALL(Loss))) - 1, 12, 31)
or
Start Date = DATE(YEAR(MAXX(ALL(Loss), Loss[dte_month_end])) - 1, 12, 31)
If you don't do this, the MAX only looks at the current row, rather than all the rows in the table.