I have a Power BI Report which contains a data flow which is automatically refreshed each morning, and a static excel which contains daily production targets.
The excel file is in a folder on a shared drive, for which I had set up a personal gateway in order to refresh. However, if I am not online the gateway is of course not active.
To try to circumvent this issue, I chose to exclude the excel file from refresh in power query editor. However, when the automatic refresh occurs each morning, I am still getting an error message that the report has failed to refresh as my personal gateway was not line.
Is there another way to avoid this issue?
Related
My problem is very easy: I'm retrieving all stocks about lithium on the ASX market from this web page.
I created a Power BI dashboard that you can visualize here.
As the source is Web I would like to refresh it every day using the Gateway connection, so I have successfully installed it on my computer that I turn on every day:
The problem is that instead of a successful daily refresh I have a daily error:
Failed to test the connection to your data source. Please retry your credentials.
Some credentials aren't validated as they are set to skip test connection.
If I enter in the Edit credentials I see the error:
Failed to update data source credentials: The credentials provided for the Web source are invalid. (Source at https://www.marketindex.com.au/commodities/lithium.)Show details
This because I'm using Authentication method: Anonymous and Privacy level setting for this data source: Public. But this is exactly what I'm using on Power BI desktop and it's working!
What am I doing wrong here?
Why the same connection is working fine on Power BI Desktop but not the same dashboard uploaded online?
EDIT: the On-premises data gateway looks installed without problems
It seems like the issues you're experiencing may be caused by the link you're using (https://www.marketindex.com.au/commodities/lithium), although I'm not sure exactly why.
For example, as has been mentioned in the Do I Need a Gateway for Web post, one can use Web.Contents instead of Web.BrowserContents in the raw M code in the Power Query Editor to avoid using a gateway. However, when I test this implementation using the link that you provide, I get an error when refreshing from Power BI online.
If I test this implementation using an HTML table from the https://tradingeconomics.com/commodity/lithium website, though, and set the data source credentials as follows, then I am able to successfully refresh the dataset from Power BI online.
I tested this with another HTML table from a random webpage and an online refresh worked as well. So it seems like there is something odd about how Power BI online interacts with the Market Index website and this is causing your issue. Potentially because of some authentication configuration on the Market Index side of things? I'm not able to help with looking further into why this is occurring, though.
Would you be able to use a different website with similar information as a data source for your report? If so, you could avoid the issues you're experiencing and avoid using a gateway in general if you wanted.
If you go this route, it seems like using the included "Scheduled refresh" setup in Power BI online works, but I will include the Power Automate scheduled refresh setup below in case this is desired instead.
Power Automate Scheduled Refresh
One can use Power Automate to "schedule" dataset refreshes instead of the included "Scheduled refresh" functionality in Power BI online. The steps are a little too long to include here, but the Refresh your Power BI dataset using Microsoft Flow article outlines the process on setting this up. There are various triggers that you can use to trigger your flow. The linked article uses one that triggers based on a SharePoint action, but there is a Recurrence trigger that can be used to trigger your flow at regular intervals/times.
Just set up your flow, make sure that it is "turned on", and your dataset will refresh at regular intervals from the cloud.
Im trying to just understand the basics of Power Bi service refreshing reports from datasets, and despite several questions on here (like this one and this one) and several microsoft articles, Im still lost.
Lets say I go on my corporate onedrive account and create a very basic excel file. Two columns, some fake names and fake numbers.
Then I close that. Open Power Bi desktop and create a very basic report where I just dump that data into a table. Also worth pointing out I bring that data in using this method to copy and paste the sharepoint path to the file (and do all the other stuff on that page as far as authenticating my work account, getting the data, etc.. I do that because in other places I read that if the onedrive path was the path as its seen on your computer, it wont work.
I then publish the report to the service online. Works great so far.
I now want to refresh my report based on a change I made to the data. So I open the onedrive excel doc, change a number, save and close.
Now no matter what I do, the report wont refresh. When I click to refresh here:
It gives me this error:
When I google that error it says I need to change the permissions, so I go to the data source settings in Power bi desktop and have done all sorts of things in various attempts. I.e. "clear permissions" or "edit permissions" and make sure it says "none", etc... etc..
What am I misunderstanding?
Lastly, I dont understand it at all, but Ive heard previously I may need a "gateway". At one point a week or two ago I had my companies IT department install a gateway and as far as I can tell its on right now:
I was told I need to select OAuth2 as the Authentication method, but where I would expect to find the menu for dataset credentials (pic below, I'd expect it to read dataset description, then gateway connection, then data source credentials) I dont even see the option:
Also obviously that warning on the screen (cant refresh because data sources dont support refresh) sounds relevant, but a- it looks like the whole error isnt even there, b- its just an excel file on onedrive, shouldn't that work?
After publishing the pbix in the Power BI you need to configure the data source credentials to use OAuth2.
I have an existing PowerBI report that imports data from an SQL Server analytics services database. This is working fine and I can schedule automatic refreshes using the Gateway provided by my organization.
I would now like to add some additional, but rarely changing data, that I only have in a local Excel file. When I do add this data, the report stops refreshing automatically and complains, that it has no gateway to refresh this Excel file.
What I would like is that Power BI is refreshing the data of the SQL Server analytics services database, but just keeps the existing Excel file without updating it. - I will upload an updated version of the PowerBI report if I need to change the data in the Excel file.
Is that possible? I couldn't find out how. I was trying to upload the Excel file to a different dataset to the Power BI service and reference this dataset in my report. Just to find out, that I cannot access a different Power BI dataset and SQL server analysis services database from the same report.
Three things I can think of
Upload the file to onedrive/sharepoint so that it's accessible online (per Dev's answer)
If the data is simple enough, you can add the data directly into PowerBI itself and skip the Excel file entirely.
You can disable the Excel file refresh so that PBI does not try and refresh(and thus access) the local Excel file. (Not sure if this will work)
I had a similar issue I came across. Yes, you can just use Enter Data to add a table, but you can only build something with less than 3000 cells, so you'd have to merge several tables if something was larger than that.
Turning off the report refresh in the suggestion above (#3) still requires a gateway, unfortunately.
I just created a dataflow and plopped the data from my csv there. You'll have to create a connection and refresh it, but you don't need to schedule a refresh there, so no need to create a gateway.
Then just link the dataflow as a source to your .pbix file and setup your gateway to point at the dataflow.
I haven't done any changes in my dataset for a while and in the past few days, the refresh and publish features (via PBI Desktop) were working just fine. However, this morning, it suddenly stopped following through the scheduled/automatic refresh which were set 5x a day every 2 hours starting 8AM. When Itrigger manual refresh, it says "Preparing for refresh" but doesn't really follow through with it, so i kept on clicking it but to no avail. After a while, I checked the Refresh History and the error I got was it's timing out.
I was trying to republish it with a newer version which was the same dataset refreshed in a different workspace. Apparently, when I publish it to a different workspace, or if I rename it and publish it to my existing shared workspace, it works. BUT I don't want to push through with this kind of workaround because I cannot afford to redo my the set up of RLS and manage permission in the dataset of my existing workspace.
Hope you can help me on this. Thanks.
Try signing in to the workspace using a browser, then use the Get Data feature (arrow button at bottom left), then choose Files / Local File and select your PBIX.
It's effectively the same results as a Publish from Power BI Desktop, but sometimes the method above works when Power BI Desktop is jammed up.
You might also have a refresh issue, not a publish issue. There's an unannounced-refresh-on-publish feature that most people don't notice:
"When you republish a dataset published from Power BI Desktop and have a refresh schedule defined, a dataset refresh is started as soon as you republish."
https://learn.microsoft.com/en-us/power-bi/create-reports/desktop-upload-desktop-files
I created a report in Powerbi desktop. The dataset is a view from a table and I used the Import method and not DirectQuery. There is a lot of data. So when I do a refresh on the PowerBi desktop, it does the refresh but takes a while. When I published my report, and scheduled a daily refresh, the daily refresh fails with the following error
Data source type: SQL
Data source error: Microsoft SQL: Query timeout expired
How can I olve this. I have tried deleting and publishing the report again and again. I also checked the data credentials field in the schedule refresh tab.
I would open the report file in Power BI Desktop and make these changes:
Navigate to the "file" menu (top-left) / Options and settings / Settings
Under Current File / Data Load / Parallel Loading ..., check the Enable parallel loading ... option
Under Current File / Privacy, choose Ignore the Privacy Levels ...
Hopefully that's enough to get you over the line.
If not then I would work on speeding the query design. From your error message it sounds like the generated query is very slow to start returning rows. I'd try to move any Filter, Group By or Choose/Remove Column steps up the Applied Steps list so they have an impact early and reduce the load on your database server.