I have some data that looks like this (more than 400 columns) :
year
ID
fake_num1
fake_num2
text1
2019
11
36 000
10'000
text, 1
2020
12
-1 275
1 000,00
text 2
Columns fake_num1 and fake_num2 are stored as text. What I'm trying to achieve is
Identify those fake numbers columns
Clean the data (e.g. remove space, columns, replace comma by points) with a for loop
I need some help with step 1. I have to identify columns fake_num1 and fake_num2, while avoiding columns like text1. I was thinking of going with regexp but maybe there is another solution.
I used part of the code here: SO regexp, however I am not sure how to proceed from there.
Dim strPattern as String: strPattern = "^[0-9]$"
will find anything that starts and ends with a number, and only has numbers (if my comprehension is correct). What's the best way to manage the cases listed in the table above ?
Please, try the next code, It considers "fake numbers columns" as ones where replacing the necessary characters makes from string a number:
Sub testMakeNumbers()
Dim sh As Worksheet, lastR As Long, lastCol As Long, i As Long, rngCol As Range
Set sh = ActiveSheet 'you can use here the necessary sheet
lastR = sh.Range("A" & sh.rows.Count).End(xlUp).row
lastCol = sh.cells(1, Columns.Count).End(xlToLeft).Column
'determine the problematic columns:
For i = 1 To lastCol
If Not IsNumeric(sh.cells(2, i).Value) And _
IsNumeric(Replace(Replace(Replace(sh.cells(2, i).Value, " ", ""), "'", ""), ",", ".")) Then
If rngCol Is Nothing Then
Set rngCol = sh.cells(2, i)
Else
Set rngCol = Union(rngCol, sh.cells(2, i))
End If
End If
Next
'replace the characters making the string as number:
With Intersect(rngCol.EntireColumn, sh.Range("A2", sh.cells(lastR, lastCol)))
.Replace ",", "."
.Replace Chr(160), ""
.Replace " ", ""
.Replace "'", ""
End With
End Sub
Related
I am trying to build regex pattern for the text like that
numb contra: 1.29151306 number mafo: 66662308
numb contra 1.30789668number mafo 60.046483
numb contra/ 1.29154056 number mafo: 666692638
numb contra 137459625
mafo: 666692638
mafo: 666692638 numb contra/ 1.29154056
Here's the pattern I could build
contra?.\s+?(\d+\.?\d+)(.+mafo.?\s+(\d+\.?\d+))?
It works fine for all the lines except the last one. How can I implement all the possibilities to include the last line too?
Please have a look at this link
https://regex101.com/r/pSThAU/1
All is OK as for contra but not as for mafo
I think the key here is to make your regexp do less and your vba do more. What I think I see here is either the word 'mafo' or 'contra' and a number following. Don't know what order or whether each is present or how many times. So you can scan each of your strings for ALL occurrences with a regexp like this:
(?:^|[^A-Z])(?:(mafo)|(contra))[^A-Z]\s*(\d*\.?\d+)
Then process it with some VBA code like this that I created in Excel:
Sub BreakItUp()
Dim rg As RegExp, scanned As MatchCollection, eachMatch As Match, i As Long, col As Long
Set rg = New RegExp
rg.Pattern = "(?:^|[^A-Z])(?:(mafo)|(contra))[^A-Z]\s*(\d*\.?\d+)"
rg.IgnoreCase = True
rg.Global = True
i = 1
Do While (Not IsEmpty(ActiveSheet.Cells(i, 1).Value))
Set scanned = rg.Execute(ActiveSheet.Cells(i, 1).Value)
col = 2
For Each eachMatch In scanned
ActiveSheet.Cells(i, col).Value = eachMatch.SubMatches(0) & eachMatch.SubMatches(1)
ActiveSheet.Cells(i, col + 1).Value2 = "'" & eachMatch.SubMatches(2)
col = col + 2
Next eachMatch
i = i + 1
Loop
End Sub
That MatchCollection object will get one item for each Match that occurs and the subMatches array contains each capturing group. You should be able write your own logic within this processing loop to interpret what was extracted. When I ran it on your data it created all the fields in blue:
Notice I added a line to your data that had two contra entries and one mafo and it found all the occurrences. You should be able to modify this to interpret the meanings.
In Power Query, I have a list of emails that includes invalid emails. I am looking to use M codes to identify and "fix" them. For example, my email list would include something like "1234.my_email_gmail_com#error.invalid.com"
I am looking for Power Query to find similar email addresses, then produce an output of a valid email. For the example above, it should be "my_email#gmail.com"
Essentially, I want to do the following:
Remove the digits at the front (number of digits varies)
Remove the "#error.invalid.com"
Replace the first underscore "_" from the right to "."
Replace the second underscore "_" from the right to "#"
I'm still new to Power Query, especially with M codes. I appreciate any help and guidance I can get.
Try the function cleanEmailAddress below:
let
cleanEmailAddress = (invalidEmailAddress as text) as text =>
let
removeLeadingNumbers = Text.AfterDelimiter(invalidEmailAddress, "."), // Assumes invalid numbers are followed by "." which itself also needs removing.
removeInvalidDomain = Text.BeforeDelimiter(removeLeadingNumbers, "#"),
replaceLastOccurrence = (someText as text, oldText as text, newText as text) as text =>
let
lastPosition = Text.PositionOf(someText, oldText, Occurrence.Last),
replaced = if lastPosition >= 0 then Text.ReplaceRange(someText, lastPosition, Text.Length(oldText), newText) else someText
in replaced,
overwriteTopLevelDomainSeparator = replaceLastOccurrence(removeInvalidDomain, "_", "."),
overwriteAtSymbol = replaceLastOccurrence(overwriteTopLevelDomainSeparator, "_", "#")
in overwriteAtSymbol,
cleaned = cleanEmailAddress("1234.my_email_gmail_com#error.invalid.com")
in
cleaned
Regarding:
"Remove the digits at the front (number of digits varies)"
Your question doesn't mention what to do with the leading . (which remains if you remove the leading digits), but your expected output ("my_email#gmail.com") suggests it should be removed. Email addresses which do not have . immediately after the leading digits, will return an error (and the logic for removeLeadingNumbers expression will need to be improved).
This seems to work too:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Valid", each Text.ReplaceRange(Text.ReplaceRange(Text.BetweenDelimiters([Column1],".","#"),Text.PositionOf(Text.BetweenDelimiters([Column1],".","#"),"_",Occurrence.Last),1,"."),Text.PositionOf(Text.ReplaceRange(Text.BetweenDelimiters([Column1],".","#"),Text.PositionOf(Text.BetweenDelimiters([Column1],".","#"),"_",Occurrence.Last),1,"."),"_",Occurrence.Last),1,"#"))
in
#"Added Custom"
I have a bunch of data which contains any number of 5-digit strings in completely inconsistent formats, and i want to extract these 5-digit strings (in bold) out. I am not bothered about strings containing less than or more than 5-digits. as an example, this is the kind of data i have in my file
Cell A1: "1. 76589 - wholesale activities. 2. 33476 - general"
Cell A2: "WHOLESALE ACTIVITIES (76589). SHIPPING (12235). REAL
ESTATE ACTIVITIES (67333)"
Cell A3: "1. 33476 General. 658709 annual road. Unknown 563"
I've tried the usual SEARCH/FIND, MIN, LEFT/RIGHT/MID functions, but am not sure how to get them to produce the result i need, and even text-to-columns wasn't giving me a clean result
thanks in advance
Here is a macro that will split your line into the columns as you requested.
The range being processed is whatever you have selected.
The results are written into the adjacent columns on the same row.
Depending on your worksheet setup, you may want to "clear out" the rows where the results are going before executing the extraction code.
You can also write code to select the data to be processed automatically. Plenty of examples on this forum.
Option Explicit
Sub Extract5Digits()
Dim R As Range, C As Range
Dim RE As Object, MC As Object, M As Object
Dim I As Long
Set R = Selection
Set RE = CreateObject("vbscript.regexp")
With RE
.Global = True
.Pattern = "\b\d{5}\b"
For Each C In R
If .test(C.Text) = True Then
I = 0
Set MC = .Execute(C.Text)
For Each M In MC
I = I + 1
C.Offset(0, I) = M
Next M
End If
Next C
End With
End Sub
Simply with Excel functions this is impossibile.
The best way for you is to use the Regex 55 library in VBA.
Let's consider this example:
+---+--------------------------------------------------------------+
| | A |
+---+--------------------------------------------------------------+
| 1 | Cell A3: "1. 33476 General. 658709 annual road. Unknown 563" |
| 2 | 33476 |
+---+--------------------------------------------------------------+
From the Excel file hit Alt + F11, then go to Tools => Reference and select "Microsoft VBScript Regular Expression 5.5".
Then you can use the following function definition:
Public Function Get5DigitsNumer(search_str As String)
Dim regEx As New VBScript_RegExp_55.RegExp
Dim matches
GetStringInParens = ""
regEx.Pattern = "[0-9]{5}"
regEx.Global = True
If regEx.test(search_str) Then
Set matches = regEx.Execute(search_str)
GetStringInParens = matches(0).SubMatches(0)
End If
End Function
At this time you can use the following code:
Sub PatternExtractor()
Range("A2").Value = Get5DigitsNumer(Range("A1"))
End Sub
which take the value of cell A1 and extract the 5 digits numer, thn the result is saved into cell A2.
At the time I don't have any idea how this code could work where the same cell contains more than one time; like "Cell A1: "1. 76589 - wholesale activities. 2. 33476 - general" in your example.
I suggest you to have a look at this answer. The pattern is different but the question is really similar to yours.
The only way that you can do it is by writing a regex in VBA. I would recommend you to look at this question.
Is that possible? Probably not? How can I then find all exact occurrences of a match and the according page numbers?
EDIT:
I have the regex working properly. What I need is for each match to get all the pages it appears on.
Example:
regex = \b\d{3}\b
123 appears on page 1,4,20
243 appear on page 3,5,7
523 appears on page 9
How can I get that information (all the pages a match occurs on?)
This is for creating some kind of index automatically.
EDIT 2:
I got a basic working version, snippet:
Set Matches = regExp.Execute(ActiveDocument.range.Text)
For Each Match In Matches
Set range = ActiveDocument.range(Match.FirstIndex, Match.FirstIndex + Len(Match.Value))
page = range.Information(wdActiveEndAdjustedPageNumber)
The problem is that Match.FirstIndex does not always point to the first character of the match in ActiveDocument.range. Word tables mess this up as ActiveDocument.range.Text contains characters that are not on the text put represent something in the table.
I think this probably fits better in SuperUser.
The answer to the question is "yes."
Selection.Information(wdActiveEndAdjustedPageNumber)
The above property in VBA will get you the page number of a selection.
Also, VBA can do some regular expression work.
This turned out to be rather complex and I can't say if my solution works for any document. The main issue is as indicated in the Question, that RegexMatch.FirstIndex can not be used to determine were the actually Match is within the MS Word Document. This is due to the fact that regex matching is done on range.Text property (String) and that string just contains different amount of characters than the range object does and hence Indexes don't match.
So my solution is for each match, I do a Find in the whole document for that match. the find methods gives a Range object from which the correct page can be determined.
In my special case a match could be the same thing also different value. Example: 343in my case would be the same as Prefix-343. A second issue was that the matches must be sorted eg 123before 324regardless which one occurs first in the document.
If you require the Sort Functionality you will also need the following to "modules":
SortDictionary Function:
http://www.cpearson.com/excel/CollectionsAndDictionaries.htm
Module "modQSortInPlace":
http://www.cpearson.com/Zips/modQSortInPlace.zip
If no sort is needed you don't need them but you need to remove the according function call SortDictionary Dict, Truefrom my code.
Now to my code. Soem parts you can remove, especially the formatting one. This is specific to my case. Also if your match is "unique", eg. not prefix or so you can simplify the code too. You will need to reference the "Microsoft Scripting Library".
Option Explicit
Sub ExtractRNumbers()
Dim Dict As Scripting.Dictionary
Set Dict = CreateObject("Scripting.dictionary")
Dim regExp, Match, Matches
Dim rNumber As String
Dim range As range
Set regExp = CreateObject("VBScript.RegExp")
regExp.Pattern = "\b(R-)?\d{2}-\d{4,5}(-\d)?\b"
regExp.IgnoreCase = False
regExp.Global = True
' determine main section, only extract R-Numbers from main section
' and not the Table of contents as example
' main section = section with most characters
Dim section As section
Dim maxSectionSize As Long
Dim sectionSize As Long
Dim sectionIndex As Integer
Dim currentIndex As Integer
maxSectionSize = 0
currentIndex = 1
For Each section In ActiveDocument.Sections
sectionSize = Len(section.range.text)
If sectionSize > maxSectionSize Then
maxSectionSize = sectionSize
sectionIndex = currentIndex
End If
currentIndex = currentIndex + 1
Next
Set Matches = regExp.Execute(ActiveDocument.Sections(sectionIndex).range.text)
For Each Match In Matches
' If the Document contains Tables, ActiveDocument.range.Text will contain
' BEL charachters (chr(7)) that probably define the table structure. The issue
' is that then Match.FirstIndex does not point to the actual first charachter
' of a Match in the Document.
' Also there are other things (unknwon) that lead to the same issue, eg.
' Match.FirstIndex can not be used to find the actual "matching word" within the
' document. Because of that below commented apporach does not work on a generic document
' Set range = ActiveDocument.range(Match.FirstIndex, Match.FirstIndex + Len(Match.Value))
' page = range.Information(wdActiveEndAdjustedPageNumber)
' Maybe there is a simpler solution but this works more or less
' the exception beign tables again. see http://support.microsoft.com/kb/274003
' After a match is found the whole document is searched using the find method.
' For each find result the page number is put into an array (if it is not in the array yet)
' Then the match is formatted properly.
' After formatting, it is checked if the match was previously already found
'
' If not, we add a new entry to the dictionary (key = formatted match, value = array of page numbers)
'
' If match was already found before (but potentially in a different format! eg R-87-1000 vs 87-1000 as example),
' all additional pages are added to the already found pages.
Set range = ActiveDocument.Sections(sectionIndex).range
With range.Find
.text = Match.Value
.MatchWholeWord = True
.MatchCase = True
.Wrap = wdFindStop
End With
Dim page As Variant
Dim pages() As Integer
Dim index As Integer
index = 0
ReDim pages(0)
Do While range.Find.Execute() = True
page = range.Information(wdActiveEndAdjustedPageNumber)
If Not IsInArray(page, pages) Then
ReDim Preserve pages(index)
pages(index) = page
index = index + 1
End If
Loop
' FORMAT TO PROPER R-NUMBER: This is specific to my case
rNumber = Match.Value
If Not rNumber Like "R-*" Then
rNumber = "R-" & rNumber
End If
' remove possible batch number as r-number
If Len(rNumber) > 11 Then
rNumber = Left(rNumber, Len(rNumber) - 2)
End If
' END FORMAT
If Not Dict.Exists(rNumber) Then
Dict.Add rNumber, pages
Else
Dim existingPages() As Integer
existingPages = Dict(rNumber)
For Each page In pages
If Not IsInArray(page, existingPages) Then
' add additonal pages. this means that the previous match
' was formatted different, eg R-87-1000 vs 87-1000 as example
ReDim Preserve existingPages(UBound(existingPages) + 1)
existingPages(UBound(existingPages)) = page
Dict(rNumber) = existingPages
End If
Next
End If
Next
'sort dictionary by key (R-Number)
SortDictionary Dict, True
Dim fso
Set fso = CreateObject("Scripting.FileSystemObject")
Dim stream
' Create a TextStream.
Set stream = fso.CreateTextFile(ActiveDocument.Path & "\" & ActiveDocument.Name & "-rNumbers.txt", True)
Dim key As Variant
Dim output As String
Dim i As Integer
For Each key In Dict.Keys()
output = key & vbTab
pages = Dict(key)
For i = LBound(pages) To UBound(pages)
output = output & pages(i) & ", "
Next
output = Left(output, Len(output) - 2)
stream.WriteLine output
Next
Set Dict = Nothing
stream.Close
End Sub
Private Function IsInArray(page As Variant, pages As Variant) As Boolean
Dim i As Integer
IsInArray = False
For i = LBound(pages) To UBound(pages)
If pages(i) = page Then
IsInArray = True
Exit For
End If
Next
End Function
I have a table with so many rows. It's structure is like this picture:
As you can see i have "or", "And" between names in columns A. How i can splite these column into twi parts?. IN that case i will have David, Tylor, Fred, Jessi, Roland in the firstcolumn and Peter, Mark, Alfered, Hovard and DAvid in the second.
Note: Please pay attention to row 2 and 5. in these rows i have 2 "or" or two "and".
Edit: I prefer to do that in Excel
What I Have Tried
As one possible solution, i have this function in vba.
Function udfRegEx(CellLocation As Range, RegPattern As String)
Dim RegEx As Object, RegMatchCollection As Object, RegMatch As Object
Dim OutPutStr As String
Dim i As Integer
i = ActiveWorkbook.Worksheets(ActiveWorksheet.Name).UsedRange.rows.Count
Set RegEx = CreateObject("vbscript.regexp")
With RegEx
.Global = True
.Pattern = RegPattern
End With
OutPutStr = ""
Set RegMatchCollection = RegEx.Execute(CellLocation.Value)
For Each RegMatch In RegMatchCollection
OutPutStr = OutPutStr & RegMatch
Next
udfRegEx = OutPutStr
Set RegMatchCollection = Nothing
Set RegEx = Nothing
Set Myrange = Nothing
End Function
This function uses Regex. but i don't know how to use that.
As I mentioned that you do not need VBA for this. An Excel formula will also do what you need.
My Assumptions
Col A has the data
You want the output in Col B and Col C
Paste this formula in Cell B1 and copy it down
=IF(ISERROR(SEARCH(" or ",A1,1))=TRUE,IF(ISERROR(SEARCH(" and ",A1,1))=TRUE,"",LEFT(A1,SEARCH(" and ",A1,1))),LEFT(A1,SEARCH(" or ",A1,1)))
and this in Cell C1 and copy it down
=IF(ISERROR(SEARCH(" or ",A1,1))=TRUE,IF(ISERROR(SEARCH(" and ",A1,1))=TRUE,"",MID(A1,SEARCH(" and ",A1,1)+5,LEN(A1)-SEARCH(" and ",A1,1))),MID(A1,SEARCH(" or ",A1,1)+4,LEN(A1)-SEARCH(" or ",A1,1)))
SNAPSHOT
(\w)+(( or | and ){0,1}(\w)+)*
Its not a coding solution, but since you did not ask for code (and because its not necessary in this case), simply do a find/replace on the words "and" and "or" to replace them with some delimiter (e.g. replace them with a comma). Then in excel, you can select the data, and split them into different columns using excels "text to columns" feature (on the data tab in excel 2007).