Authentication error using Yahoo Finance in Power BI when updating data - powerbi

When trying to autoupdate stock values using Yahoo Finance link from a csv file using this:
Source = Csv.Document(
Web.Contents(
"https://query1.finance.yahoo.com/v7/finance/download/"
& Comp
& "?period1=1022112000&period2="
& LastDate
& "&interval=1d&events=history"
),
[Delimiter = ",", Columns = 7, Encoding = 1252, QuoteStyle = QuoteStyle.None]
Extra info: Comp and lastdate are custom parameters, in which comp fetches all company stock data and lastdate records the most recent stock date.
I keep getting authentication errors everytime I try to access the data using either anonymous or user login. What can I do?
Thanks

Please check what value you provide to the URL string. Probably there is some error.
You can check this example with function:
let
Source = (STOCKNAME, StartDat, EndDat) => let
Source = Csv.Document(Web.Contents("https://query1.finance.yahoo.com/v7/finance/download/"&STOCKNAME&"?period1="&StartDat&"&period2="&EndDat&"&interval=1d&events=history&includeAdjustedClose=true"),[Delimiter=",", Columns=7, Encoding=1252, QuoteStyle=QuoteStyle.None]),
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"Date", type date}, {"Open", type text}, {"High", type text}, {"Low", type text}, {"Close", type text}, {"Adj Close", type text}, {"Volume", Int64.Type}})
in
#"Changed Type"
in
Source
Query:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WCgkPCVLSUTI0M7E0MzY1MjCAcsyNDM2AnFidaCVPvxBnVFEIx8LAwgCkJBYA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [TICKER = _t, Start = _t, End = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"TICKER", type text}}),
#"Invoked Custom Function" = Table.AddColumn(#"Changed Type", "GetStock", each GetStock([TICKER], [Start], [End])),
#"Expanded GetStock" = Table.ExpandTableColumn(#"Invoked Custom Function", "GetStock", {"Date", "Open", "High", "Low", "Close", "Adj Close", "Volume"}, {"GetStock.Date", "GetStock.Open", "GetStock.High", "GetStock.Low", "GetStock.Close", "GetStock.Adj Close", "GetStock.Volume"})
in
#"Expanded GetStock"

Related

How to remove the part of text in one column that matches the value of another column?

I have a table with a column of email addresses and another column of unique IDs. For some reason, some of the values in the email address are prefixed with the unique IDs so I need to do some cleanup of the email address column by removing the unique ID prefix from the email address column. The length of the unique ID can vary, so it's not as straight forward as the only method I know of simply removing the first x amount of characters:
email
unique ID
george#hotmail.com
79fsdfv8v2657f842356j0
784ret3956w35497er834r62mary#hotmail.com
784ret3956w35497er834r62
34gfng6w5h6pauly#outlook.com
34gfng6w5h6
Resulting column:
george#hotmail.com
mary#hotmail.com
pauly#outlook.com
The solution has to be in m code, not DAX. Any ideas?
you can do it on the power query side...
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("dYzNCgIhEIBfJea8RKgz6m3fQzxIqVtpE667y759EARBdP5+nIMcueU4TtxruJbjmesBBtA2zZe0mlUQ6mSUkEi3E/jBgTaqxS4t0iZRWR2bkaqRqKHtP5s/7nskVU6PTBtO9AxL2UdeemG+f+IvDt6/AA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"email " = _t, #"unique ID" = _t]),
#"Duplicated Column" = Table.DuplicateColumn(Source, "email ", "email - Original"),
#"Replaced Value" = Table.ReplaceValue(#"Duplicated Column",each [unique ID],"",Replacer.ReplaceText,{"email "}),
#"Renamed Columns" = Table.RenameColumns(#"Replaced Value",{{"email ", "Result"}}),
#"Reordered Columns" = Table.ReorderColumns(#"Renamed Columns",{"unique ID", "email - Original", "Result"})
in
#"Reordered Columns"
sample file as solved above...
I found this answer and their solution solved my problem:
https://community.powerbi.com/t5/Power-Query/Dynamically-remove-string-from-one-column-if-string-is-contained/td-p/1809532
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
CLEAN = Table.ReplaceValue(Table.TransformColumns(Table.AddColumn(Source, "All ICD-10 Replaced",
each if Text.Contains([#"All ICD-10"], [#"ICD-10"])
then Text.Replace([#"All ICD-10"], [#"ICD-10"], "")
else [#"All ICD-10"]),
{{"All ICD-10 Replaced", Text.Trim, type text}}),
" "," ",Replacer.ReplaceText,{"All ICD-10 Replaced"})
in
CLEAN

pbi m query to concatenate a field

Not well versed in PBI formulas but kinda know what I'm looking for. I have a comment field that can be long so what I'm trying to do is
if text.length([comment] > 45) then text.range(text.combine([comment],"..."),45) else [comment]
Some of the comments field will be null as well. I've tried different variations of this and just can't see to get it right. Appreciate any help. Thanks.
Note, M is case sensitive, so text.length() Null and text.combine will not work. You have to use Text.Length() , Text.Combine() , null, etc.
Try this:
= try if Text.Length([comment])>45 then Text.Start([comment],45)&"..." else [comment] otherwise null
You can use add that as part of a new custom column:
let Source = #table({"comment"},{{"abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"},{"Bus"},{null}, {"Car Log"}}),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"comment", type text}}),
#"Add Custom Column" = Table.AddColumn( #"Changed Type", "Custom", each try if Text.Length([comment])>45 then Text.Start([comment],45)&"..." else [comment] otherwise null)
in #"Add Custom Column"
Or transform the existing column:
let Source = #table({"comment"},{{"abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"},{"Bus"},{null}, {"Car Log"}}),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"comment", type text}}),
#"Transform Column" = Table.TransformColumns(#"Changed Type",{{"comment", each try if Text.Length(_)>45 then Text.Start(_,45)&"..." else _ otherwise null, type text}})
in #"Transform Column"

Power BI: Expression.Error: We cannot convert the value null to type Logical

In the following code, I'm getting an error on line 10 ("Expanded Table Column1"):
let
Source = SharePoint.Files("https://microsoft.sharepoint.com/teams/Fake_Folder/", [ApiVersion = 15]),
#"Added Custom" = Table.AddColumn(Source, "find file", each if Text.Contains([Name], "Fake_Name")
then 1 else 0),
#"Filtered Rows" = Table.SelectRows(#"Added Custom", each ([Name] <> null)),
#"Filtered Rows1" = Table.SelectRows(#"Filtered Rows", each ([find file] = 1)),
#"Filtered Hidden Files1" = Table.SelectRows(#"Filtered Rows1", each [Attributes]?[Hidden]? <> true),
#"Invoke Custom Function1" = Table.AddColumn(#"Filtered Hidden Files1", "Transform File (6)", each #"Transform File (6)"([Content])),
#"Removed Other Columns1" = Table.SelectColumns(#"Invoke Custom Function1", {"Transform File (6)"}),
#"Expanded Table Column1" = Table.ExpandTableColumn(#"Removed Other Columns1", "Transform File (6)",
Basically, I'm selecting a file that matches the name "Fake_Name", and then expanding that file. and I'm getting the error:
We cannot convert the value null to type Logical
Prior to the error, the file is being located, it's being filtered, and then when it needs to be expanded as a table, it runs into a null value. How do I fix it when I can't go to the error and the file itself doesn't have the nulls I'm looking for?
Found the issue: "Sample File 6" (another file referred to in this query) had an issue which caused it to be unloaded, which caused the issue seen here.

Concatenating dynamic column name to table - Power Query

I am trying to create lists of data using dynamic columns, fetching the column name and appending it to the table.
Here is the problem as this step is throwing error:
Expression.Error: We cannot apply operator & to types Table and Text.
Details:
Operator=&
Left=[Table]
Right=Test1
Code:
let
Source = Excel.Workbook(File.Contents("C:\Users\Sivakumar A\Desktop\Power BI\test1.xlsx"), null, true),
Sheet2_Sheet = Source{[Item="Sheet2",Kind="Sheet"]}[Data],
#"Promoted Headers" = Table.PromoteHeaders(Sheet2_Sheet, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"Test1", type text}, {"Test2", Int64.Type}, {"Test3", Int64.Type}, {"Test4", Int64.Type}}),
#"Col Names" = Table.ColumnNames(#"Changed Type"),
#"Generate" = List.Generate(() => [i=-1,x= Table.FromList(#"Col Names",Splitter.SplitByNothing())],
each [i]<List.Count(#"Col Names"),
each [
i=[i]+1,
x=#"Promoted Headers"&#"Col Names"{i}
],
each [x])
in
Generate
I need to extract the column names and append it to the talble and execute in loop
Any help is appreciated.
My requirement is to append the column names dynamically to the table using list.generate and generate the list of values out of it.
I was able to resolve the issue using Double Quotes 3 times.
"Table.Distinct(Table.FromList(#"&"""Sheet2 (2)"""&"["&#"Col Names"{i}&"], Splitter.SplitByNothing(), null, null, ExtraValues.Error))"

Newly added column in cvs source data (combined folder) - not appearing when refreshing in power query section in Power BI

Still stumbling my way through Power BI.
I have added a new column in just a few of the files in the folder that I had already pulled in and combined but this column is not appearing once refreshed. Despite the endless research, I am at a loss as to how to fix this in advanced editor.
Your assistance would be greatly appreciated.
Code below:
Source = Folder.Files("C:\Users\Sarah\OneDrive\FEEDLOT\APS Files\Original Files"),
#"Filtered Hidden Files1" = Table.SelectRows(Source, each [Attributes]?[Hidden]? <> true),
#"Invoke Custom Function1" = Table.AddColumn(#"Filtered Hidden Files1", "Transform File", each #"Transform File"([Content])),
#"Renamed Columns1" = Table.RenameColumns(#"Invoke Custom Function1", {"Name", "Source.Name"}),
#"Removed Other Columns1" = Table.SelectColumns(#"Renamed Columns1", {"Source.Name", "Transform File"}),
#"Removed Errors1" = Table.RemoveRowsWithErrors(#"Removed Other Columns1", {"Transform File"}),
#"Expanded Table Column1" = Table.ExpandTableColumn(#"Removed Errors1", "Transform File", Table.ColumnNames(#"Transform File"(#"Sample File"))),
#"Changed Type" = Table.TransformColumnTypes(#"Expanded Table Column1",{{"Source.Name", type text}, {"Tag Number", type text}, {"Electronic ID", type text}, {"NLIS", type any}, {"Date", type datetime}, {"Live Weight (kg)", type number}, {"Draft", type any}, {"Condition Score", type any}, {"Notes", type any}}),
#"Renamed Columns" = Table.RenameColumns(#"Changed Type",{{"Source.Name", "APS Source File Name"}, {"Tag Number", "Visual ID"}}),
#"Split Column by Delimiter" = Table.SplitColumn(#"Renamed Columns", "APS Source File Name", Splitter.SplitTextByEachDelimiter({"-"}, QuoteStyle.Csv, true), {"APS Source File Name.1", "APS Source File Name.2"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"APS Source File Name.1", type text}, {"APS Source File Name.2", type text}}),
#"Removed Columns" = Table.RemoveColumns(#"Changed Type1",{"APS Source File Name.2"}),
#"Renamed Columns2" = Table.RenameColumns(#"Removed Columns",{{"APS Source File Name.1", "APS Source File Name"}})
First of all, check twice you are reading the correct file and backup the current script. Then, I would try two approaches:
Delete the steps until you find the one that causes the problem. Then, fix the step causing the issue and check if at the end of the pipeline the columns appear.
If this does not solve the issue, I would go with the second approach:
Create a new PBIX, and one-by-one re-implement all the steps until you find the one causing the new columns to disappear.
Don't try to fix this in Advanced Editor alone. Instead, step through the code and inspect the result after each step. That will show you where the problem is.
Without seeing your data it is not possible to tell you where in your code you need to make changes, but you will be able to see that if you step through the actions one by one.