I have created a Flow in Power automate, have used a Refresh a Power BI dataset component , there is no issue in terms of functionality as such and I am able to refresh the dataset successfully - so far so good (till the time the dataset gets refreshed successfully)
I have added a send email component post the dataset refresh, Now, the thing is the 'Refresh a dataset' component doesn't wait till the time the dataset is completely refreshed, it just triggers the refresh with an output value as Succeeded and proceed towards sending an email while the dataset refresh is still in progress
Now, when the dataset refresh fails due to some reason - the Power Automate flow has already completed and has lost the track of the refresh
How can I make it wait till the time the dataset refresh has completed successfully or has failed and then send an email accordingly?
I have tried below but the outcome is always Success
Solved: Get alert when refresh a dataset fail - Power Platform Community (microsoft.com)
The Refresh action sends an API call to the Power BI REST API. The Refresh Dataset endpoint does not respond with an outcome indication. It just responds with whether or not the refresh request was accepted.
There are two other endpoint options Get Refresh History and Get Refresh Execution Details which you would need to monitor for completion of your refresh. I'm not saying it's impossible to implement something with Power Automate to do this, but honestly, you will have a better outcome just writing it yourself.
Power BI's lack of admin and monitoring tooling is by far its biggest shortcoming--especially surrounding refresh management. The sooner you realize you will not get out of it what you want and just accept that you're going to have to roll your own, the sooner you'll get what you need.
Related
My problem is very easy: I'm retrieving all stocks about lithium on the ASX market from this web page.
I created a Power BI dashboard that you can visualize here.
As the source is Web I would like to refresh it every day using the Gateway connection, so I have successfully installed it on my computer that I turn on every day:
The problem is that instead of a successful daily refresh I have a daily error:
Failed to test the connection to your data source. Please retry your credentials.
Some credentials aren't validated as they are set to skip test connection.
If I enter in the Edit credentials I see the error:
Failed to update data source credentials: The credentials provided for the Web source are invalid. (Source at https://www.marketindex.com.au/commodities/lithium.)Show details
This because I'm using Authentication method: Anonymous and Privacy level setting for this data source: Public. But this is exactly what I'm using on Power BI desktop and it's working!
What am I doing wrong here?
Why the same connection is working fine on Power BI Desktop but not the same dashboard uploaded online?
EDIT: the On-premises data gateway looks installed without problems
It seems like the issues you're experiencing may be caused by the link you're using (https://www.marketindex.com.au/commodities/lithium), although I'm not sure exactly why.
For example, as has been mentioned in the Do I Need a Gateway for Web post, one can use Web.Contents instead of Web.BrowserContents in the raw M code in the Power Query Editor to avoid using a gateway. However, when I test this implementation using the link that you provide, I get an error when refreshing from Power BI online.
If I test this implementation using an HTML table from the https://tradingeconomics.com/commodity/lithium website, though, and set the data source credentials as follows, then I am able to successfully refresh the dataset from Power BI online.
I tested this with another HTML table from a random webpage and an online refresh worked as well. So it seems like there is something odd about how Power BI online interacts with the Market Index website and this is causing your issue. Potentially because of some authentication configuration on the Market Index side of things? I'm not able to help with looking further into why this is occurring, though.
Would you be able to use a different website with similar information as a data source for your report? If so, you could avoid the issues you're experiencing and avoid using a gateway in general if you wanted.
If you go this route, it seems like using the included "Scheduled refresh" setup in Power BI online works, but I will include the Power Automate scheduled refresh setup below in case this is desired instead.
Power Automate Scheduled Refresh
One can use Power Automate to "schedule" dataset refreshes instead of the included "Scheduled refresh" functionality in Power BI online. The steps are a little too long to include here, but the Refresh your Power BI dataset using Microsoft Flow article outlines the process on setting this up. There are various triggers that you can use to trigger your flow. The linked article uses one that triggers based on a SharePoint action, but there is a Recurrence trigger that can be used to trigger your flow at regular intervals/times.
Just set up your flow, make sure that it is "turned on", and your dataset will refresh at regular intervals from the cloud.
I am using Azure Logic App workflows to refresh a Power BI dataset every 5 minutes. The workflow was working for several weeks without any issues . But since last 2 days the dataset refresh is failing (Message - Another refresh running at same time). I have attached the picture on the refresh status. Though it shows the alternate refresh status as Success , the Power BI dashboard shows the same number all the time.
Is there a way to fix this the steps I tried - Disabled and enabled the flow, Deleted the flow and created again but its the same problem.
Also there are other reports that do not have this issue. I am using a Service account for the API connection .Is there a way to check if there is another refresh running at same time. Thanks for any inputs.
.
At present, I have a dashboard connected to reports as seen below:
It would appear, even though all 3 are connected to the same live azure analysis services instance, the report on the right does not refresh live, but only once every several days.
Is there a setting that I missed that would permit connecting to azure analysis services in a more live way than simply connecting to it?
If your dataset is configured for a live or DirectQuery connection, datasets are refreshed approximately each hour or when interaction with the data occurs. You can manually adjust the refresh frequency in the Scheduled cache refresh option in the Power BI service.
When you refresh data, you are updating the data in the dataset that is stored in Power BI from your data source. This refresh is a full refresh and not incremental.
For a live connection to Analysis Services, the dataset definition comes from Analysis Services directly.
For more details, refer “Data refresh in Power BI”.
Hope this helps.
I am looking at retrieving data from an API in Power BI.
I would like to avoid processing all of the data again when I run a refresh. Is there a way to add the new data to the existing data in the dashboard as a delta upload instead of a full refresh every time?
Thanks for any help.
You need to buy Premium for this. Power BI Premium supports Incremental refresh (see announcement and documentation).
With Incremental refresh you define two parameters, RangeStart and RangeEnd, and policies how to refresh the deltas. The values for these parameters are managed by the service. Then first refresh will be slow, because it will load all the data, but next refreshes will only load the data for the period after the last refresh.
I have implemented Power BI embedded in a web app with direct query using Azure SQL as data source.
The Azure SQL database is being updated by webjobs and if I leave open the Power BI embedded web app I don't see the visuals refreshing with the new data unless I trigger a query for example changing tab or filtering with a slicer.
In the documentation I found the following:
"If there is no user interaction in a visualization, like in a dashboard, data is refreshed automatically about every fifteen minutes."
Do I understand correctly that an open visual in my case should be refreshing without need of user interaction?
Can you point out to the reason for it not to be updating automatically? Also do you know a way to control the time of the refresh with direct query without user interaction more exactly than the "...about every fifteen minutes..."
When inspecting the connection properties on Power BI desktop I have made sure it indicates "Direct Query".
From my understanding the embedded report won't refresh automatically. However, if you're using the Power BI JS framework (https://github.com/Microsoft/PowerBI-JavaScript) to embed your report from a Workspace Collection, then you can use a refresh() method on the report object to manually get the latest data, provided your report is using Direct Query.
This method is only present in version 2.2.0 of the framework and was then removed in the latest version (currently 2.2.1) while further testing around billing is performed (see https://github.com/Microsoft/PowerBI-JavaScript/commit/5230b2f96b10a1104efecdffe78255b9788526b8).
However, in my testing I found the session count remained unaffected by the refresh method. You can refresh up to intervals of 15 seconds (a limit set by the server). This may change however, given the method was removed in 2.2.1, but using 2.2.0 seems to work currently.
Here's a quick and dirty example which will refresh the report every minute within the allocated session:
powerbi.embed(reportContainer, embedConfig);
report = powerbi.get(reportContainer);
window.setInterval(function () {
report.refresh();
}, 60 * 1000);
If the session expires (after 1 hour currently) then a new JWT will need to be requested and the report needs to be reloaded with the new token.
You may want to implement some checks around the session expiry if you plan to keep the report open for more than the allotted session time.