I'm trying to import data from a web page to Power BI Desktop. The URL is the following:
https://casa.sapo.pt/Venda/Apartamentos/Maia/?sa=13
It contains data about housing prices, characteristics, etc.
The query returns an empty table, although if I browse to the page with any browser and without authentication or login, I see it contains the data I'm looking to analyse so I guess the publisher has disabled somehow the query I'm trying to make.
Does anyone know a workaround?
Power BI / Power Query gives quite disappointing results against that fairly typical page structure. It only seems to work well against "Web 1.0" style HTML tables. There are so few of those around (outside of Wikipedia) that it is almost pointless ...
You could try Chris Webb's ExpandAll function (link below) that will expand out all the data available.
http://blog.crossjoin.co.uk/2014/05/21/expanding-all-columns-in-a-table-in-power-query/
Then you can exclude all the non ".Text" columns and merge them together to get all the text from the page in one column - but it's hard to make sense of it. You also lose a lot of the HTML content eg links, image URLs.
Related
The PowerBI connector doesn't work, since it defaults to the IE browser, and our IE browser is locked down by our IT department (can't connect to smartsheet site). So, I need to use the "web" API option to get my smartsheet data. I have the data coming in, but I'm brand new at this and I'm having trouble organizing it. I have 8 columns that are repeating as rows, and the cell data contains a bunch of metadata values that I don't want. How do I get this so the columns become columns again, with the cell data as rows (with just the face values) and not repeating?
screenshot of data in PowerBI
I think you need to read a good part of the documentation in order to get used to the responses you get from Smarsheet, in brief you get a Json data structure.
First read, https://smartsheet-platform.github.io/api-docs/#how-to-read-a-sheet-response
Then you need to make the json connection to PowerBI like this example, https://www.mssqltips.com/sqlservertip/4621/using-power-bi-with-json-data-sources-and-files/
I have an Existing report in Power BI with Oracle Data Source from which I had directly access the Table.
But now I need to Change the Data Source of the Report using SSAS.
By Using SSAS, we can access the oracle Tables and deploy the SSAS in Azure Service.
Now through this Azure Service we can access the Data.
My Question is Without any Changes in the Report, Can we Simply Change the DataSource? Because I'm Showing Lot of Graphs in that Report.
Since I'm New to Power BI , Is this possible?
There will likely be an unpredictably large amount of other issues (character encoding, date formatting, etc.) that you'll have to work through.
The way I might approach this problem is, if the tables are exactly identical, or you didn't make any changes within power query (like removing columns, merging tables, etc.) you may be able to modify the M code within the advanced editor, and try to swap the data sources to see if it works.
Go into "Transform Data"
Select the table you want to modify
Click the "Advanced Editor" icon in the ribbon.
Here, I imported an excel file, but for you, it should show some kind of "AnalysisServices" line. I don't have an SSAS database to connect to so I can't validate. Try replacing this line with the connection string to your SSAS datasource and see if it works.
Save, apply update, and see if it works?
Again, this is not really advisable, but if you want to give it a try, and the data sources are identical, this is how I might approach the problem before just re-making the report whole-cloth.
I'm trying to build a simple report in Power BI based upon data published on a website.
Here is what I want to achieve
This website publishes data for COVID cases in the country.
The number are just the current numbers, without any time-series.
I want to fetch these numbers from this website daily and build a report on
top of it (with time series kind of analysis).
So I fetch these numbers (Get Data > Web > URL) and get this into a query I then add
a custom column with a timestmap (M's DateTime.LocalNow() function)
and get this data with the required timestamp.
Now I want to refresh this query daily, so that I get daily results in this query.
6. As expected, PBI simply overwrites the existing rows with new data,
with the latest timestamp (my custom column).
I tried few things like:-
Creating a new query and appending data to it, it doesn't seem to work, existing data gets over-written (maybe the way I have created the new query).
Explored incremental refresh functionality, it doesn't seem to fit my use case.
Tried looking at other similar posts, none seem to help me resolve this.
Questions:-
Is there a simple workaround to circumvent this (point#7) and have PBI append new data instead of overwriting existing data.
Am i correct on point#2 above (incremental refresh)?
Appreciate any pointers. Thanks in advance!
There is no simple workaround within Power BI.
Power BI is not designed to be used as a database where you store historical data. It's designed to connect to data and create reports from that, so you'll need to store the daily data somewhere external.
There are tons of ways to store the data. E.g., you could save them as CSVs in a folder that Power BI loads from or you could write them to a database table and connect to that.
Edit: That said, there is a non-simple workaround if this is something you really must do.
Though not recommended, you can use incremental refresh to trick Power BI into doing what you want.
I have a URL that returns a json object with everything I need for my power bi embedded report. I get the data for the report by adding a new web data source and pasting the URL in. a few transformations later and tada! sexy report. the report shows lots of charts and graphs etc... however I need to be able to change the datasource URL depending on who is looking at it.
The report shows data for a single organization. You can only look at it if you're in that organization. how can I pass an organizations ID when embedding the report so that the datasource will show different data?
for example if my datasource is defined in the originating pbix as
Json.Document(Web.Contents("http://www.testdata.com/api/json?orgId=1"))
how can I change it to
Json.Document(Web.Contents("http://www.testdata.com/api/json?orgId=2"))
when I'm pull the report to embed on a page?
I know you can filter data but that means I have to make the datasource URL pull ALL the data which would be huge and intensive just to have bi filter out something.
In short, I'm embedding a report on a website and tat report's only way to get data is via a json endpoint. That endpoint requires the org id of the user so how do I pass it to bi which in turn uses it in the data source url?
Your only option for this scenario is to pull all the required data into your dataset. Then you can use either Role Level Security (RLS) or the new JS API to filter the data for each user.
You should probably look at an Azure SQL data source as a more efficient, flexible and scalable back-end for PBI Embedded.
I am using Power BI API.
I've got a dataset with some tables and rows.
From Power BI API Console I don't have any issue when retrieving datasets or tables.
However the PUT verb on a table resource to update its schema always returns a 504 - Proxy request timed out
It's the first time I use Apiary IO so it might be its problem and not Power BI update, but that leads me to some questions:
Is there any workaround to test Power BI with, for example, Fiddler? I can type the url and body but I will need an Authorization header with the OAuth2 token if I'm not mistaken. How can I get that? ApiaryIO seems to hide it.
As per Update Schema Documentation the URL with the resource is https://api.powerbi.com/v1.0/myorg/datasets/{myDatasetId}/tables/{myTableName}
and the verb is a PUT. What is then the meaning of the "name": "???" parameter that goes in the JSON body? Is it the table's name or something else? I am assuming it's the table name but it seems redundant as I am already accessing the resource {myTableName} as per the given URL.
And my last related question is how to rename a specific table's column without modifying its data? This is what I'm trying to achieve by updating the schema but I don't understand how does Power BI know what column I am trying to rename.
Thank you!
Sorry that you're having trouble. You can get a token in two ways -the right way is to create an app in AAD (here's how). The wrong way ;) is to open the Power BI.com service, in a browser then open fiddler, then press F5 to reload. You should be able to see the Access Token in various requests. If you register an app, you can plug in your App's information in one of the samples we have https://powerbi.microsoft.com/developers, see client app or web app.
The name you provide in the table is the friendly human readable name that appears in the UI when you're building a report. Without it the system is unusable by humans :).
Let me get back to you on #3.
Calling PUT table will attempt to save upgrade the table without loosing any data (unless you removed columns). If it can't, it will return a conflict error. If you still want to update the table schema, you would have to delete the rows and call PUT table again. There is currently no direct way to rename a column. PUT table would treat it like a delete and add for that column. You would loose the data in that column but not the whole table.