I need to do a one time load of all the rows from a Sharepoint list into Sql Server. I am having trouble with parsing through the Get Items control. Especially for fields that are arrays.
Has anybody tried this before?
Thanks
Barry
Related
Hello I am working on a dashboard that needs to combine 2 tables: sales table, sales order table. Two two tables come from different sources: sales table from SAP BW and sales order table through a dataflow. Both tables have a number of applied steps that delete, reformat and rename the columns. I then want to append the Sales Order Table to the Sales table. In the desktop version everything works even if I hit "refresh all" in the Power Query Editor. When I publish the report to the service and refresh, I am getting an error saying: The key didn't match any rows in the table
Full Error Message:
After troubleshooting for the last couple of days I'm pretty sure that the issue has to do with my column names being renamed. However its strange that the append would work in desktop but not in service.
My main questions are:
Can you append tables after multiple applied transformation steps?
Can the tables both refresh daily or does one or the other have to be static?
Does the order of the table queries in query editor effect the order in which tables are loaded or does everything load all at once? Can this order affect my append query?
Any suggestions or help would be greatly appreciated. I am 99% of the way to launching this new report but this refresh issue is driving me crazy.
You can append tables after multiple applied transformation steps
Both tables can refresh daily
The order of the table queries in query editor doesn't matter. Power BI will determine the order in which they are loaded. E.g. if you want to append a table to another table, that other table is loaded first.
Thanks for answering those questions Peter. I concluded that the issue had something to do with appending a dataflow. I switched to a new data source from SAP BW and its working now. What's interesting is that the dataflow is still being imported and refreshed daily so the issue had to do with just the append.
I have a Power-BI dashboard that was created some time ago. It’s data source is a SharePoint-Online list. Since the Power-BI dashboard was created, several new fields have been added to the SharePoint list. Now I am being asked to add a new page to the dashboard that reports on those new fields. However, I have not found a way to get the existing Power-BI list/dataset to show the new fields.
Refreshing the data does refresh the values, but refresh does not add the new fields.
I’ve spend the last 4 hours looking on the internet for a solution. The only thing I have been able to do so far is to attach the list again with a different name- the new fields DO show up when I do this. (I can’t just replace the older Power-BI list/dataset because there have been several calculated columns and measures added.)
I can work with this and create the report, but is this the only way? It doesn’t seem like it should be.
Any help would be appreciated! Thank you!
(I'm using Power BI April 2021 and Sharepoint Online)
So, it looks like there's no good answer to this issue. I found that adding another instance of the reference Sharepoint list, that included the new columns, did work (however inelegant). That seems to be the best direct answer for times when the older pbix file must continue to be used.
What I ended up doing, though, was to create a new separate pbix file which included the latest version of the Sharepoint List. This was the best solution for my organization since it will allow us to be more focused on the specific manufacturing processes involved.
Thanks to #Jon and #Alejandro for their efforts to help!
If you have access to PowerAutomate you could refresh the dataset creating a flow so that given a certain time (say, once or twice a day) the dataset gets refreshed with the new created items.
Otherwise if you are working with the service version of Power BI you can program a refresh of the dataset directly from the workspace going to the settings of the dataset. You would have to have a gateway set for that which could be in personal mode or not.
Also if you want to update the data in the service version you could do it manually too in the workspace.
I am using a Sharepoint onlne list as a Source for Power BI. There are 67 sharepoint online lists , whom i have to append together, and then map it to an excel in my Local Disk.
I have given the Table.Buffer command which i helps to load the table faster. But still it takes long time for the table to load.
Can anyone help me with this Issue or provide an alternative
Unfortunately, the way Excel loads the data you cannot really make it faster. You can try disabling "Background Data" (go to Query Options > Data Load > deselect 'Allow data preview to download in the background') option but that is limited performance enhancement. Here are some articles on the subject:
https://www.myonlinetraininghub.com/excel-forum/power-query/any-way-to-speed-up-really-slow-refresh-times-in-power-query
https://powerpivotpro.com/2017/07/power-query-refresh-speeds-suck/
But there is a better way!!! You can connect your PowerBI directly to the Sharepoint lists without the Excel step and it will be much faster!
I am using the Power Bi Report Server to create a report that uses the Sharepoint Online List. Has anyone figured out some way or a workaround where we can use a Sharepoint Online List and create a scheduled refresh? I have tried importing that data into an Excel file on Sharepoint and also tried to import it from CSV file that gets the update using Flows, but I can't find a way where we can have scheduled refresh. Is there any other way apart from but the data from that list in excel file on a shared directory?
Any ideas would be appreciated.
I have to migrate some legacy data from stand-alone sql server database to sharepoint list.
I'm going to use programmatic approach and write a code that communicates with sharepoint list asmx web service.
Are there some "data transformation wizards" to simplify such a task or a better approach to port legacy data from sql server database to sharepoint list?
Thank you in advance!
Being one time operation, I would not worrry about Best Practice but would consider what's the fastest way to do it.
You can use Excel 2010 (I have not tested it with Excel 2007) export data to Sharepoint 2010. Here are the high level steps:
Import data from SQL Server using DATA Tab in the ribbon
Excel would automatically create a TABLE
Now you can prepare the data for Export to Sharepoint. Here, you can remove unwanted columns, add new columns remove unwanted rows, arrange columns etc.
While being in the Table, access the "Export Table To Sharepoint List" functionality to publish you data to Sharepoint. More information about this is available at: http://office.microsoft.com/en-gb/excel-help/export-an-excel-table-to-a-sharepoint-list-HA010131472.aspx
It is quick! but let;s be aware of the limitations:
1. It cannot publish data to a list which already exists
2. It will not create a content type for the exported list. The columns are directly attached to the list.
If you want greater control over the migration, programming may be the way to go unless someone has a better idea in this great forum!