Text.Trim Syntax Meaning - powerbi

I'm working in a Power BI query, trying to trim whitespace from text.
Looking at Microsoft's M reference, I came across the Text.Trim syntax:
Text.Trim(text as nullable text, optional trimChars as any) as nullable text
I couldn't figure out how to plug it into my query code correctly (where it would actually work) so I did some more searching and came across this:
#"Trimmed Text" = Table.TransformColumns(#"Removed Other Columns",{},Text.Trim),
...which doesn't look anything like Microsoft's syntax but works fine for me as I insert it into my code like this:
let
Source = Banding,
#"Removed Other Columns" = Table.SelectColumns(Source,{"Segment", "Granular Band"}),
#"Trimmed Text" = Table.TransformColumns(#"Removed Other Columns",{},Text.Trim),
#"Removed Duplicates" = Table.Distinct(#"Trimmed Text", {"Granular Band"})
in
#"Removed Duplicates"
My problem is that I don't understand what the line's syntax meaning is. I understood Microsoft's example as meaning basically, trim THIS; where THIS is the text I want trimmed. Pretty straightforward.
But I don't know what the syntax meaning of the line that actually works is. I understand that it says to transform table columns (Table.TransformColumns); but I don't know if the reference to the previous line (#"Removed Other Columns") serves as any real "input" for the Text.Trim or if {} is a reference to all columns in the table, or (more importantly to me) how I would reference specific columns. (I've tried a few approaches for specifying columns and failed every time.) I also don't understand why I don't need any arguments following Text.Trim (like in Microsoft's example).
If someone would translate what the line is "saying" in a manner I can understand, I'd sure appreciate it.

The generated code:
#"Trimmed Text" = Table.TransformColumns(#"Removed Other Columns",{},Text.Trim),
means that you most probably selected all columns and then you selected Transform - Format - Trim.
If you would have selected 1 or more columns, then the names of these columns and the required operations would have been between the {}, like in
= Table.TransformColumns(#"Removed Other Columns",{{"SomeText", Text.Trim}})
Any function that is invoked within Table.TransformColumns, gets the column values automatically supplied from Table.TransformColumns, so in this case: the first argument for Text.Trim (text as nullable text).
The other arguments will use the default value, i.c. space, so by default all leading and trailing spaces are removed.
If you want to use other arguments of Text.Trim within the Table.TransformColumns context, then you need to adjust the code and supply the keyword "each", and use an _ as placeholder for the column values.
For example the next code removes leading and trailing spaces and semicolons:
= Table.TransformColumns(#"Removed Other Columns",{{"SomeText", each Text.Trim(_, {" ",";"})}})

Related

Remove columns by name based on pattern

How can I remove a large number of columns by name based on a pattern?
A data set exported from Jira has a ton of extra columns that I've no interest in. 400 Log entries, 50 Comments, dozens of links or attachments. Problem is that they get random numbers assigned which means that removing them with hardcoded column names will not work. That would look like this and break as the numbers change:
= Table.RemoveColumns(#"Previous Step",{"Watchers", "Watchers_10", "Watchers_11", "Watchers_12", "Watchers_13", "Watchers_14", "Watchers_15", "Watchers_16", "Watchers_17", "Watchers_18", "Watchers_19", "Watchers_20", "Watchers_21", "Watchers_22", "Watchers_23", "Watchers_24", "Watchers_25", "Watchers_26", "Watchers_27", "Watchers_28", "Log Work", "Log Work_29", "Log Work_30", "Log Work_31", "Log Work_32", ...
How can I remove a large number of columns by using a pattern in the name? i.e. remove all "Log Work" columns.
The best way I've found is to use List.FindText on Table.ColumnNames to get a list of column names dynamically based on target string:
= Table.RemoveColumns(#"Previous Step", List.FindText(Table.ColumnNames(#"Previous Step"), "Log Work")
This works by first grabbing the full list of Column Names and keeping only the ones that match the search string. That's then sent to RemoveColumns as normal.
Limitation appears to be that FindText doesn't offer complex pattern matching.
Of course, when you want to remove a lot of different patterns, having individual steps isn't very interesting. A way to combine this is to use List.Combine to join the resulting column names together.
That becomes:
= Table.RemoveColumns(L, List.Combine({ List.FindText(Table.ColumnNames(L), "Watchers_"), List.FindText(Table.ColumnNames(L), "Log Work"), List.FindText(Table.ColumnNames(L), "Comment"), List.FindText(Table.ColumnNames(L), "issue link"), List.FindText(Table.ColumnNames(L), "Attachment")} ))
SO what's actually written there is:
Table.RemoveColumns(PreviousStep, List.Combine({ foundList1, foundlist2, ... }))
Note the { } that signifies a list! You need to use this as List.Combine only accepts a single argument which is itself already a List of lists. And the Combine call is required here.
Also note the L here instead of #"Previous Step". That's used to make the entire thing more readable. Achieved by inserting a step named "L" that just has = #"Promoted Headers".
This allows relatively maintainable removal of multiple columns by name, but it's far from perfect.

Is there any way that I can do format matching within a column in powerBI? ( something similar Fuzzy)

I have a column look like as below.
DK060
DK705
DK715
dk681
dk724
Dk716
Dk 685 (there is a space after Dk).
This is obviously due to human error. Is there any way that I can ensure the format is correct based on the specified format which is two uppercase DK followed by three digits?
Or Am I being too ambitious!!??
Go to the power query editor. Select advance editor and paste this 2 steps
#"Uppercase" = Table.TransformColumns(#"Source",{{"Column", Text.Upper, type text}}),
#"Replace Value" = Table.ReplaceValue(#"Uppercase"," ","",Replacer.ReplaceText,{"Column"})
Note: be sure to replace the "Source" statement into the Uppercase sentence for your previuos step name if needed.
So you will have something like this:
This is the expected result:

What is wrong with my Power BI query (using a parameter)?

I'm brand new to using PBI but as far as I can tell, I should be able to substitute a parameter as part of a Direct Query in place of a hard-coded variable...ie
let
Source = Sql.Database("NAMEOFDB", "CMUtility", [Query="sp_get_residentsinfo "& home_name]),.....
instead of
let
Source = Sql.Database("NAMEOFDB", "CMUtility", [Query="sp_get_residentsinfo 'NAME OF HOME'"]),...
However, the parameter-included version just says
DataSource.Error: Microsoft SQL: Incorrect syntax near 'House'.
Details:
DataSourceKind=SQL
DataSourcePath=NAMEOFDB;CMUtility
Message=Incorrect syntax near 'House'.
Number=102
Class=15
"House" is the currently - assigned last word of the home_name variable. What have I done wrong?
PS - I have surmised that I shouldn't need the extra & at the end of the parameter, as I'm not adding anything else to the query, but even with both &s it still doesn't work.
The type of your parameters is text. In SQL, text literals must be quoted, i.e. sp_get_residentsinfo 'NAME OF HOME', but the statement build by you is sp_get_residentsinfo NAME OF HOME.
You should use Text.Replace to escape single quotes in the parameter's value and append a quote before and after it.

Power Query - Table.TranformColumns

I'm trying to add leading zeros into a column in power query call JobCodes. I know I can do this by adding a new column using Text.Start([JobCodes],5,"0"), but I don't want to add a new column and go back to remove the column I don't need. I want to be able to do this in one step using Table.TransformColumns function. Is this possible?
Code:
Table.TransformColumns(#"Changed Type", each Text.PadStart([JobCodes],5,"0"))
Error:
Expression.Error: We cannot convert a value of type Function to type List.
Details:
Value=Function
Type=Type
Your syntax is just a bit off.
I think this is what you want:
= Table.TransformColumns(#"Changed Type",{{"JobCodes", each Text.PadStart(_, 5,"0")}})
The error is because it was expecting a list of columns that you want to transform (notice the {{...}} above.
The easiest wat to get the syntax right is to use the GUI to do a transformation and then just edit the function a bit. For example, you could use Format > Add Prefix which would give you the following step (assuming you choose to prefix 000).
= Table.TransformColumns(#"Changed Type", {{"JobCodes", each "000" & _, type text}})
Just take out the "000" & _ and put in the transformation you actually want.
Or
= Table.ReplaceValue( Source, each [JobCodes] ,each Text.PadStart(Number.ToText([JobCodes]),5,"0") ,Replacer.ReplaceValue,{"JobCodes"})
in Replace
The correct expresion is:
= Table.TransformColumns(#"Changed Type",{{"JobCodes", each Text.PadStart(Text.From( _), 6, "0"), type text}})

How to split column in Power Query by the first space?

I am using Power Query and have a column called LandArea; example data is "123.5 sq mi". It is of data type text. I want to remove the "sq mi" part so I just have the number value, 123.5. I tried the Replace function to replace "sq mi" with blank but that doesn't work because it looks at the entire text. So I tried to use Split where I split it on the space and it generated this formula below, and it did create a new column, but with null for all values. The original column still had "123.5 sq mi".
Table.SplitColumn(#"Reordered Columns1","LandArea",Splitter.SplitTextByDelimiter(" ", QuoteStyle.None),{"LandArea.1", "LandArea.2"})
When just splitting at the left-most delimiter:
Table.SplitColumn(#"Reordered Columns1","LandArea",Splitter.SplitTextByEachDelimiter({" "}, QuoteStyle.None, false),{"LandArea.1", "LandArea.2"})
I have also tried changing to QuoteStyle.Csv. Any idea how I can get this to work?
Use this to create a custom column:
= Table.AddColumn(
#"Reordered Columns1",
"NewColumn",
each Text.Start([LandArea],Text.PositionOf([LandArea]," "))
)
UPDATE: Every one appears to have "sq mi"
= Table.AddColumn(#"Changed Type", "Custom", each Text.Replace([LandArea]," sq mi",""),type number)
Hope it helps.
This is what I ended up using:
Table.AddColumn(#"Reordered Columns1", "LandArea2",
each Text.Start([LandArea], Text.PositionOf([LandArea], "sq")-1))
I avoided trying to find whitespace.