Concatenate strings in Power BI with Query editor - powerbi

I trying to make function with Advanced Query editor. I make simple function and that function work well.Below you can see how is look like that code:
(page as number) as table =>
let
Source = Web.BrowserContents("https://www.neptun.mk/pocetna/categories/TV_AUDIO_VIDEO/televizori.nspx?items=100&page=" & Number.ToText(page)),
#"Extracted Table From Html" = Html.Table(Source, {{"Column1", ".product-list-item__content\-\-title"}, {"Column2", ".HappyCard"}, {"Column3", ".newPriceModel"}, {"Column4", "[width=""192""]", each [Attributes][src]?}}, [RowSelector=".product-list-item-grid"]),
#"Changed Type" = Table.TransformColumnTypes(#"Extracted Table From Html",{{"Column1", type text}, {"Column2", type text}, {"Column3", type text}, {"Column4", type text}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each "https://www.neptun.mk"),
#"Merged Columns" = Table.CombineColumns(#"Added Custom",{"Custom", "Column4"},Combiner.CombineTextByDelimiter("", QuoteStyle.None),"Merged"),
#"Split Column by Delimiter" = Table.SplitColumn(#"Merged Columns", "Column2", Splitter.SplitTextByDelimiter(" ", QuoteStyle.Csv), {"Column2.1", "Column2.2", "Column2.3", "Column2.4"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"Column2.1", type text}, {"Column2.2", type text}, {"Column2.3", Int64.Type}, {"Column2.4", type text}}),
#"Split Column by Delimiter1" = Table.SplitColumn(#"Changed Type1", "Column3", Splitter.SplitTextByDelimiter(" ", QuoteStyle.Csv), {"Column3.1", "Column3.2", "Column3.3", "Column3.4"}),
#"Changed Type2" = Table.TransformColumnTypes(#"Split Column by Delimiter1",{{"Column3.1", type text}, {"Column3.2", type text}, {"Column3.3", Int64.Type}, {"Column3.4", type text}}),
#"Removed Other Columns" = Table.SelectColumns(#"Changed Type2",{"Column1", "Column2.3", "Column3.3", "Merged"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Other Columns",{{"Column1", "ImageURL"}, {"Column2.3", "RegularPrice"}, {"Column3.3", "DiscountedPrice"}, {"Merged", "Title"}})
in
#"Renamed Columns"
Most important part of this piece of code is third line which begin with `Source =
Web.BrowserContents("https://www.neptun.mk/pocetna/categories/TV_AUDIO_VIDEO/televizori.nspx?items=100&page=" & Number.ToText(page)),` Actually here I want to add some additional string and concatenate with this piece of code.
In purpose to do that first I want to declare one more variable which is call items
(page as number) as table =>
(items as number) as table =>
let
Source = Web.BrowserContents("https://www.neptun.mk/pocetna/categories/TV_AUDIO_VIDEO/televizori.nspx?items=100&page=" & Number.ToText(page)),
But now I have problem how to concatenate variable items into below part of code before part " & Number.ToText(page).I try with this line but is not work well:
Source = Web.BrowserContents("https://www.neptun.mk/pocetna/categories/TV_AUDIO_VIDEO/televizori.nspx?items="& Number.ToText(items),page=" & Number.ToText(page)),
So can anybody help me how to fix this code and concatenate into this line of code ?

Related

Power Query : Split Table Every n Columns

I have many columns of same table
Table A
Monthly Plan
Weekly plan
Actual
Monthly_Plan1
Weekly_plan 2
Actual_3
A
B
C
D
E
F
I want them as :
Monthly Plan
Weekly plan
Actual
A
B
C
D
E
F
I can not create a separate table and append it because there are so many columns and I cant create too many tables.
Here you go.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclRQ0lFyAhHOIMIFRLiCCDel2FgA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Monthly Plan " = _t, #"Weekly plan " = _t, #"Actual " = _t, #"Monthly_Plan1 " = _t, #"Weekly_plan 2 " = _t, Actual_3 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Monthly Plan ", type text}, {"Weekly plan ", type text}, {"Actual ", type text}, {"Monthly_Plan1 ", type text}, {"Weekly_plan 2 ", type text}, {"Actual_3", type text}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each List.Split( Record.ToList( _), 3)),
#"Removed Other Columns" = Table.SelectColumns(#"Added Custom",{"Custom"}),
#"Expanded Custom" = Table.ExpandListColumn(#"Removed Other Columns", "Custom"),
#"Extracted Values" = Table.TransformColumns(#"Expanded Custom", {"Custom", each Text.Combine(List.Transform(_, Text.From), "|"), type text}),
#"Split Column by Delimiter" = Table.SplitColumn(#"Extracted Values", "Custom", Splitter.SplitTextByDelimiter("|", QuoteStyle.Csv), {"Monthly", "Weekly", "Actual"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"Monthly", type text}, {"Weekly", type text}, {"Actual", type text}})
in
#"Changed Type1"
Add a custom column
Remove other columns
Expand
Extract values concatenating with a |
Split the column and rename
If you don't care about row order this also works
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
Combo = List.Split(Table.ColumnNames(Source),3),
#"Added Custom" =List.Accumulate(Combo, #table({"Column1"}, {}),(state,current)=> state & Table.Skip(Table.DemoteHeaders(Table.SelectColumns(Source, current)),1))
in #"Added Custom"

Power Query Transform data within Group/Buckets

I want to transform a column in Power query. Only the transformation should be applied within the group based on a condition. This is my data.
Here in the above table, I just want to transform Office based column to all 1 if any Office-based column is set to 1 on the particular ID group. But all the Office based column value is 0 on the particular ID group, it should not transform the column.
My expected result would be,
It would be fine, If an additional column can have the transformed column.
try this
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", type text}, {"Item", type text}, {"Home Based", Int64.Type}, {"Office Based", Int64.Type}, {"Amount", Int64.Type}}),
// find all IDs with 1 in Office Based
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each ([Office Based] = 1)),
#"Removed Other Columns" = Table.SelectColumns(#"Filtered Rows",{"ID"}),
#"Removed Duplicates" = Table.Distinct(#"Removed Other Columns"),
//merge that back in
#"Merged Queries" = Table.NestedJoin(#"Changed Type", {"ID"}, #"Removed Duplicates", {"ID"}, "Table2", JoinKind.LeftOuter),
#"Expanded Table2" = Table.ExpandTableColumn(#"Merged Queries", "Table2", {"ID"}, {"ID.1"}),
// if there was a match convert to 1 otherwise take original number
#"Added Custom" = Table.AddColumn(#"Expanded Table2", "OfficeBased2", each try if Text.Length([ID.1])>0 then 1 else [Office Based] otherwise [Office Based]),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Office Based", "ID.1"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"OfficeBased2", "OfficeBased"}})
in #"Renamed Columns"
or the more compact version:
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Added Custom" = Table.AddColumn(Source,"Custom",(i)=>Table.SelectRows(Source, each [ID]=i[ID]) [Office Based]),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Office Based2", each if List.Contains([Custom],1) then 1 else [Office Based]),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom1",{"Custom", "Office Based"})
in #"Removed Columns"
The first method probably works best for large data sets
Here's another method:
Group by ID
Apply the Table.TransformColumns operation to each subtable
Then re-expand
let
Source = Excel.CurrentWorkbook(){[Name="Table2"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", type text}, {"Item", type text}, {"Home Based", Int64.Type}, {"Office Based", Int64.Type}, {"Amount", Int64.Type}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"ID"}, {
{"Xform", (t)=>Table.TransformColumns(t, {
{"Office Based", each if List.ContainsAny(t[Office Based],{1}) then 1 else 0}}) ,
type table
[ID=nullable text, Item=nullable text, Home Based=nullable number, Office Based=nullable number, Amount=nullable number]}
}),
#"Expanded Xform" = Table.ExpandTableColumn(#"Grouped Rows", "Xform",
{"Item", "Home Based", "Office Based", "Amount"},
{"Item", "Home Based", "Office Based", "Amount"})
in
#"Expanded Xform"

How to count duplicate values with multiple columns in Power BI by row wise

Consider:
I have four columns (A1, A2, A3 & A4) and I want to count the same/duplicate values in these four columns by grouping Index column.
For example, if "Index 1" has found the value in "A1" and the same value exists next to A2 column then it should remove it. If it’s not next to A1 column then it should stay. For example, "1 index" can take only one unique value in the four columns.
I would use Pivot and Unpivot to get this result.
Starting with a mock dataset:
I was able to transform it to this:
Here is the advanced query for your review:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("ZZDdCoMwDIXfpddezG3u51mKF9FJG2Zt0Anu7ZfUdWQUSs7hfIeU1lpTm8rASB5Yv6etrDmqWGNhJ5V1w0ujc4lyQ3BTxA5C+OGLCjCW/MrmCUSgdFozvbEZIXQPicP6x+5sNswjBuznOCknnfrAAQmffeS5oEtX75oa8lnkcX8wLe+0YXBM2w8=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [ID = _t, A1 = _t, A2 = _t, A3 = _t, A4 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}, {"A1", type text}, {"A2", type text}, {"A3", type text}, {"A4", type text}}),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Changed Type", {"ID"}, "Attribute", "Value"),
#"Filtered Rows" = Table.SelectRows(#"Unpivoted Other Columns", each [Value] <> null and [Value] <> ""),
#"Removed Duplicates" = Table.Distinct(#"Filtered Rows", {"Value", "ID"}),
#"Pivoted Column" = Table.Pivot(#"Removed Duplicates", List.Distinct(#"Removed Duplicates"[Attribute]), "Attribute", "Value"),
#"Replaced Value" = Table.ReplaceValue(#"Pivoted Column",null,"",Replacer.ReplaceValue,{"A1", "A2", "A3", "A4"})
in
#"Replaced Value"
The idea is that you will unpivot your columns so that it is possible to take advantage of 'remove duplicates' functionality. Then pivot the dataset back to its original form.
If you want to close the gaps by shifting values to the left, you will need to do a little more work. While the data is unpivoted, create a new index over the subgroups. This is an intermediate level task, Curbal has a good example (https://www.youtube.com/watch?v=7CqXdSEN2k4). Discard your 'Attribute' column from the unpivot and use your new subgroup index instead.
Here's the advanced editor for the shift:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("ZZDdCoMwDIXfpddezG3u51mKF9FJG2Zt0Anu7ZfUdWQUSs7hfIeU1lpTm8rASB5Yv6etrDmqWGNhJ5V1w0ujc4lyQ3BTxA5C+OGLCjCW/MrmCUSgdFozvbEZIXQPicP6x+5sNswjBuznOCknnfrAAQmffeS5oEtX75oa8lnkcX8wLe+0YXBM2w8=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [ID = _t, A1 = _t, A2 = _t, A3 = _t, A4 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}, {"A1", type text}, {"A2", type text}, {"A3", type text}, {"A4", type text}}),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Changed Type", {"ID"}, "Attribute", "Value"),
#"Filtered Rows" = Table.SelectRows(#"Unpivoted Other Columns", each [Value] <> null and [Value] <> ""),
#"Removed Duplicates" = Table.Distinct(#"Filtered Rows", {"Value", "ID"}),
#"Grouped Rows" = Table.Group(#"Removed Duplicates", {"ID"}, {{"Data", each Table.AddIndexColumn(_, "AttributeRenumber", 1, 1), type table [ID=nullable number, Attribute=text, Value=text, AttributeRenumber=text]}}),
#"Removed Columns" = Table.RemoveColumns(#"Grouped Rows",{"ID"}),
#"Expanded Data" = Table.ExpandTableColumn(#"Removed Columns", "Data", {"ID", "Attribute", "Value", "AttributeRenumber"}, {"ID", "Attribute", "Value", "AttributeRenumber"}),
#"Add Prefix" = Table.TransformColumns(#"Expanded Data",{{"AttributeRenumber", each "A" & Number.ToText(_) , type text}}),
#"Removed Columns1" = Table.RemoveColumns(#"Add Prefix",{"Attribute"}),
#"Pivoted Column" = Table.Pivot(#"Removed Columns1", List.Distinct(#"Removed Columns1"[AttributeRenumber]), "AttributeRenumber", "Value"),
#"Replaced Value" = Table.ReplaceValue(#"Pivoted Column",null,"",Replacer.ReplaceValue,{"A1", "A2", "A3"})
in
#"Replaced Value"

Getting this error: "Expression.Error: The column 't_count' of the table wasn't found. Details: t_count" while executing cummulative sum

Hi I am trying to create a Cummulative sum for "t_count" field in Power BI but everytime I try to run the this formula: List.Sum(List.Range(#"Added Index"[t_count],0,[Hire_Count])) I am getting error that t_count field was not found in the table. Can anyone help me in creating cummulative sum without getting this error? I am sharing full code from advance editor.
let
Source = Excel.Workbook(File.Contents("C:\Users\rabi.jaiswal\Desktop\hr_analytics\EMPLOYEE_ATTRITION_DATA.xlsx"), null, true),
Sheet1_Sheet = Source{[Item="Sheet1",Kind="Sheet"]}[Data],
#"Promoted Headers" = Table.PromoteHeaders(Sheet1_Sheet, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"Emp_Id", Int64.Type}, {"Emp_Name", type text}, {"email", type text}, {"Join_Date", type date}, {"Joined_As", type text}, {"Department", type text}, {"Manger_Name", type text}, {"Current_Designation", type text}, {"Current_Designation_Start_Date", type date}, {"Work Hours", Int64.Type}, {"Standard_Work_Hours", Int64.Type}, {"Term_Date", type date}}),
#"Removed Columns" = Table.RemoveColumns(#"Changed Type",{"Emp_Id", "Emp_Name", "email", "Joined_As", "Department", "Manger_Name", "Current_Designation", "Current_Designation_Start_Date", "Work Hours", "Standard_Work_Hours"}),
#"Sorted Rows" = Table.Sort(#"Removed Columns",{{"Join_Date", Order.Ascending}}),
#"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "Index", 1, 1, Int64.Type),
#"Renamed Columns" = Table.RenameColumns(#"Added Index",{{"Index", "Hire_Count"}}),
#"Added Conditional Column" = Table.AddColumn(#"Renamed Columns", "t_count", each if [Term_Date] = null then 0 else 1),
#"Added Custom" = Table.AddColumn(#"Added Conditional Column", "Term_total", each List.Sum(List.Range(#"Added Index"[t_count],0,[Hire_Count])))
in
#"Added Custom"
your index starts with one, so try
#"Added Custom" = Table.AddColumn(#"Added Conditional Column", "Term_total", each List.Sum(List.FirstN(#"Added Conditional Column"[t_count],[Index]))),
if your index started from zero, it would be
#"Added Custom" = Table.AddColumn(#"Added Conditional Column", "Term_total", each List.Sum(List.FirstN(#"Added Conditional Column"[t_count],[Index]+1))),

Power BI: Total no. of records (S.No) get aggregated on performing union of CSV

I have 40-50 csv files for different cities. I would want to show the number of records across time filtered by city and hashtag.
The 'S.No' got aggregated and is showing the sum of records (I dont understand why did it shows the sum, shouldnt it be individual record numbers when performing Union?) i.e. Even if individual files have S.No from 1-50, the final table should show S.No from 1-2500. but that is not the case here.
Below image shows the data in query editor.
This image shows the table with aggregated sums in 'Data table view'
As we see, the S.No is shown aggregated.
UPDATE:
Pasting the query from Advance Editor:
let
Source = Folder.Files("C:\Users\Desktop\data collection\twitter_new_count"),
#"Filtered Hidden Files1" = Table.SelectRows(Source, each [Attributes]?[Hidden]? <> true),
#"Invoke Custom Function1" = Table.AddColumn(#"Filtered Hidden Files1", "Transform File from twitter_new_count", each #"Transform File from twitter_new_count"([Content])),
#"Renamed Columns1" = Table.RenameColumns(#"Invoke Custom Function1", {"Name", "Source.Name"}),
#"Removed Other Columns1" = Table.SelectColumns(#"Renamed Columns1", {"Source.Name", "Transform File from twitter_new_count"}),
#"Expanded Table Column1" = Table.ExpandTableColumn(#"Removed Other Columns1", "Transform File from twitter_new_count", Table.ColumnNames(#"Transform File from twitter_new_count"(#"Sample File"))),
#"Changed Type" = Table.TransformColumnTypes(#"Expanded Table Column1",{{"Source.Name", type text}, {"", Int64.Type}, {"ttext", type text}, {"date", type datetime}, {"isretweet", type logical}, {"retweetcount", Int64.Type}, {"favoritecount", Int64.Type}, {"score", Int64.Type}, {"term", type text}, {"city", type text}, {"country", type text}, {"duplicate", type logical}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each true),
#"Renamed Columns" = Table.RenameColumns(#"Filtered Rows",{{"", "S.No"}}),
#"Inserted Date" = Table.AddColumn(#"Renamed Columns", "Date.1", each DateTime.Date([date]), type date),
#"Inserted Time" = Table.AddColumn(#"Inserted Date", "Time", each DateTime.Time([date]), type time),
#"Renamed Columns2" = Table.RenameColumns(#"Inserted Time",{{"date", "Datentime"}, {"Date.1", "Date"}}),
#"Added Index" = Table.AddIndexColumn(#"Renamed Columns2", "Index", 0, 1),
#"Removed Columns" = Table.RemoveColumns(#"Added Index",{"Index"}),
#"Added Index1" = Table.AddIndexColumn(#"Removed Columns", "Index", 1, 1),
#"Renamed Columns3" = Table.RenameColumns(#"Added Index1",{{"Index", "Row Number"}}),
#"Inserted Hour" = Table.AddColumn(#"Renamed Columns3", "Hour", each Time.Hour([Datentime]), Int64.Type),
#"Filtered Rows1" = Table.SelectRows(#"Inserted Hour", each true),
#"Renamed Columns4" = Table.RenameColumns(#"Filtered Rows1",{{"Hour", "Hour of Day"}})
in
#"Renamed Columns4
I'm pretty sure those S.No values in the data table view are not aggregated, but rather they are just sorted differently than in the query editor view. Check the total number of rows to be sure.