I am looking for a formula or at least a solution to the following: how to automatically save past data upon refreshing the query in Power BI?
Context: I have a column named "End Date" containing different dates.
All these dates are of course updated every time I refresh the query.
The problem is that these end dates are manually modified in the CRM from where I pull the data; thus when refreshing, I get the latest "end dates" and lose the previous date that was written there.
I would like to somehow be able to store/keep/save the previous dates - the purpose is to count how many times these "end dates" have been modified.
Attached is a picture with the column. Please let me know if what I wrote is clear enough. I will try to reexplain better.
https://ibb.co/vvmYZNt
Best regards,
Denis
Related
This is what I hope to be a very simple issue, I'm just having a hard time putting the right search terms together in order to find the answer.
Basically, I want to preserve the data from the last refresh before the data is refreshed again, in order to compare the difference.
Example:
I have a basic web scrape that runs off and grabs the latest stock price for Microsoft:
What I want to be able to do during the refresh is to first copy the current value (283.85) to a new column and then refresh the data, so that I have a side by side current and previous price.
Really tried to find an answer, but I don't think I'm using the correct terminology.
I have never used this method. Would it be easier to add a date column to your current table and make it your record table? That way you can do a comparison and visuals from your data.
If you really want separate tables you could update your table with the date column and then write a table query to get your latest stock price according to date
THis is an example of what I think i need to do
I would like to ask some modeling advise I cannot solve myself:
I am using Power BI to visualize the time machinery is out of order.
The source is a register of equipment not functioning, with a start date and end date (note that there is no end date if the machine is not fixed yet).
I would like to show the time (hours, percentage, etc) that the machinery is out of order, filter for a specific period /date (e.g. month).
So I have 2 date columns: ‘’Start out of order’’ and ‘’Back in order’’
I do have a date table, which I usually would connect to all the date variables. However, since I am working with a Start and End date. This does not give the result I am looking for.
Any help is very much appreciated!
Kind regards,
Link to my Power BI FILE:
https://wetransfer.com/downloads/83ca3850392967d0d42a5cc71f4352c420200213160932/eb7353
Stijn
I am not sure how you would like to visualise your data, but this is what I managed to do:
create a daysdiff column with
Daysbetween = IF(ISBLANK(TF_Eventos
[End out of order]);DATEDIFF(TF_Eventos[Start out of
order];TF_Eventos[TODAY];HOUR);DATEDIFF(TF_Eventos
[Start out of order];TF_Eventos[End out of order];HOUR))
This creates your column to check difference between Dates.
Then create a separate column with your Date. In this case I copied the Start out of order date, since I thought you might wanted to be able to filter for the start dates. Then simply create a relationship between your newly created Date column and your start out of order date.
Doing so lets you create a visual with the daysbetween (in this case portrayed in hours) and your start dates. Now just simply add a slicer and you can filter on date.
Hope this helps
I am looking into utilising PowerBI to identify time saved due to various Projects. People will add the projects to a Sharepoint List which then feeds into PowerBI.
PROJECTs Table:
Project Tite, Desc, Hours/Month Saved, StartDate, EndDate, Repeat? (T/F)
[Some Projects only save a fixed 10 or so hours, others save time per month (indicated by the Repeat Column)]
I've created two measures, RUNTIME determining how long the project has run in months ((TodayDate - StartDate)/30) as well as TIMESAVED which is the total hours saved from that specific project (RUNTIME*Hours/Month Saved).
Whilst this works, it has a pretty big limitation. When selecting a range, say 01/01/2017 - 01/01/2018, any projects with a start date before that range are excluded. However these maybe on-going, meaning the time saved by this project during the range needs to be added.
I've attempted to find a solution to this, however I keep getting stuck at requiring the the filter dates from the slicer, however I'm not certain this is possible. I need those projects with on-going savings to have the savings during the period given to be counted as well.
Possible alternative maybe to create a Month/Year column per Month/Year with a custom formula per column to determine that projects Hours saved for that Month/Year however this seems inefficient, at that point back to Excel might be better.
Any ideas / suggestions would be greatly appreciated, currently running through any ideas to solve but keeps coming back to needing that value specified by the filter. Cheers in advance for any advice tackling this :)
See also: https://community.powerbi.com/t5/Desktop/Re-occuring-Savings-over-Time-with-Time-Date-Slicer/m-p/346100
Unfortunately, there is no current simple solution to this problem out-of-the-box with Power Bi. All of the slicers seem to handle dates as a single point in time. They suffer in that if you are dealing with any items that span a Start and End date (like your projects, and most of my data examples) they only take one of the dates as the input. The slicers need to accept an optional end date in our case and then perform a simple date span overlap logic to determine the items that match.
I tried to solve your problem with out-of-the-box Power Bi Desktop slicers and a custom visual Timeline Slicer I found at the store with no luck earlier this month. Out of frustration, I posted a question in the Power Bi forums for suggestions.
The final suggestion from the forums I got was to "use two Filters at Filter pane". But I am not satisfied with this answer.
The Timeline Slicer code is open source and when I get more time (ha ha), I would like to make this change to the Timeline Slicer and publish it back to the repository for everyone to use.
I will monitor this question and the forum to see if a solution emerges in the future.
You can use Timeline Storyteller. you can create your time line and add a couple Slicers for Start and End. It will split by day the dates and you won't miss any data.
I have created a powerpivot model include in the image below. I am trying to include the "IncurredLoss" value and have it sliced by time. Written Premium is in the fact table and is displaying correctly. I am aiming for IncurredLoss to display in a similar fashion
I have tried the following solutions:
Add new related column: Related(LossSummary[IncurredLoss]). Result: No data
DAX Summary Measure: =CALCULATE(SUM(LossSummary[IncurredLoss])). Result: Sum of everything in LossSummary[IncurredLoss] (not time sliced)
Simply adding the Incurred Loss column to the Pivot Table panel. Result: Sum of everything in LossSummary[IncurredLoss] (not time sliced)
A few other notes:
LossKey joins LossSummary to PolicyPremiumFact
Reportdate joins PolicyPremiumFact to the Calendar.
There is 1 row in LossSummary per date and Policy. LossKey contains this information and is the PK on that table.
Any ideas, clarifications or pointers are most certainly welcome. Thank you!
The related column should work. I was able to get it to work in both Excel 2016 and Power BI Desktop. Rather than bombarding you with questions, I'll try and walk through how I would troubleshoot further, in the hopes it gets you to a solution faster:
First, check the PolicyPremiumFact table inside Power Pivot and see if the IncurredLossRelated field is blank or not. If it is consistently blank, then the related column isn't working. The primary reason the related column wouldn't work is if there's a problem with your relationships. Things I would check:
Ensure that the relationships are between the fields you think they are between (i.e. you didn't accidentally join LossKey in one table to a different field in the other table)
Ensure that the joined fields contain the same data (i.e. you didn't call a field LossKey, but in fact, it isn't the LossKey at all)
Ensure that the joined fields are the same data type in Power Pivot (this is most common with dates: e.g. joining a text field that looks like a date to an actual date field may work, but not act as expected)
If none of the above are the problem, it doesn't hurt to walk through your data for a given date in Power Pivot. E.g. filter your PolicyPremiumFact table to a specific date and look at the LossKeys. Then go the LossSummary table and filter to those LossKeys. Stepping through like this might reveal an oversight (e.g. maybe the LossKeys weren't fully loaded into your model).
If none of the above reveals anything, or if the related column is not blank inside Power Pivot, my suggestion would be to try a newer version of Excel (e.g. Excel 2016), or the most recent version of Power BI Desktop.
If the issue still occurs in the most recent version of Excel/Power BI Desktop, then there's something else going on with your data model that's impacting the RELATED calculation. If that's the case, it would be very helpful if you could mock up your file with sample data that reproduces the problem and share it.
One final suggestion I have is to consider restructuring your tables before they arrive in your data model. In your case, I'd recommend restructuring PolicyPremiumFact to include all the facts from LossSummary, rather than having a separate table joined to your primary fact table. This is what you're doing with the RELATED field to some extent, but it's cleaner to do before or as your data is imported into Power Pivot (e.g. using SQL or Power Query) rather than in DAX.
Hope some of this helps.
I have 40 million rows in my dataset. Each day I may get an extra 100 rows. Obviously I don't want to have to import the whole 40 million each time I do a data refresh. Is it possible to do an incremental refresh where only the new rows are added?
I don't think incremental update as you describe it is possible yet.
It looks like you can push rows with Power BI REST API, if you're happy to switch to that.
However, you might find this workaround useful:
Split your table and query into two: where date <= 'somedate' and where date >'somedate'
Add an "empty query", use Table.Combine to join your two subtables. Use this as your main table.
Whenever you need to refresh, only refresh the second query (the one with where date >'somedate').
Every once in a while, when that second query starts taking a long time, change somedate to the current date and do a full refresh.
The feature has now been implemented and is called Incremental refresh. Currently it is a premium only feature.