I have a horribly formated, tab delimited, "CSV" that I'm trying to clean up.
I would like to quote all the fields; currently only some of them are. I'm trying to go through, tab by tab, and add quotes if necessary.
This RegEx will give me all the tabs.
\t
This RegEx will give me the tabs that do not END with a ".
\t(?!")
How do I get the tabs that do not start with a "?
Generally for these kinds of problems, if it's a one time occurrence, I will use Excels capabilities or other applications (SSIS? T-SQL?) to produce the desired output.
A general purpose regex will usually run into bizarre exceptions and getting it just right will often take longer and is prone to missed groups your regex didn't catch.
If this is going to happen regularly, try to fix the problem at the source and/or create a special utility program to do it.
Use negative lookbehind: (?<!")\t
For one shots like this I usually just write a little program to clean up the data, that way I also can add some validation to make sure it really has converted properly after the run. I have nothing against regex but often in my case it takes longer for me figure out the regex expression than writing a small program. :)
edit: come to think about it, the main motivator is that it is more fun - for me at least :)
Related
I'm having issues with writing out an expression to block some words.
This is my current code.
I am currently just using regex101.com to test it.
(^[GgƓɠḠḡǴǵĜĝǦǧĞğĢģǤǥĠġ](?:[^a-zA-Z]*)([ÂâÅåÀàÁáÃãÄäEeAaÆæ4#]+\s{0,30})([.*\S*]{0,1}$)|(?:[^a-zA-Z]*)[ÝýŶŷŸÿỸỹYy].)
I'm needing this to find the word "gay" but if someone writes "gay man" with a space, it doesn't even pickup the word "gay". I'm just trying to figure out why the space allows the word, I tried moving things around and adding what I could that might make sense but nothing seems to click here.
I want to share this post to you https://stackoverflow.com/a/1732454/4867612
I'm not sure I understood correctly, but with only this [GgƓɠḠḡǴǵĜĝǦǧĞğĢģǤǥĠġ][ÂâÅåÀàÁáÃãÄäEeAaÆæ4#][ÝýŶŷŸÿỸỹYy]
in https://regex101.com/r/jYpHJS/1 you can detect if the word gay is in text
I am trying to write a function which removes websites from a piece of text. I have:
removeWebsites<- function(text){
text = gsub("(http://|https://|www.)[[:alnum:]~!#$%&+-=?,:/;._]*",'',text)
return(text)
}
This handles a large set of the problem, but not a popular one, i.e something of the form xyz.com
I do not wish to add .com at the end of the above regex, as it limits the scope of that regex. However I tried writing some more regexex like:
gsub("[[:alnum:]~!#$%&+-=?,:/;._]*.com",'',testset[10])
This worked, but it also modified email ids of the form abc#xyz.com to abc#. I don't want this, so I modified it to
gsub("*((^#)[[:alnum:]~!#$%&+-=?,:/;._]*).com",'\\1',testset[10])
This left the email ids alone but stopped recognising websites of the form xyz.com
I understand that I need some sort of a set difference here, of the form of what was explained here but I was not able to implement it (mainly because I was not able to completely understand it). Any idea on how I go about solving my problem?
Edit: I tried negative lookaheads:
gsub("[[:alnum:]~!#$%&+-=?,:/;._](?!#)[^(?!.*#)]*.com",'',testset[10])
I got a 'invalid regex' error. I believe a little help in correcting may get this to work...
I can't believe it. There actually is a simple solution to it.
gsub(" ([[:alnum:]~!#$%&+-=?,:/;._]+)((.com)|(.net)|(.org)|(.info))",' ',text)
This works by:
Start with a space.
Put all sorts of things, except an '#' in.
end with a .com/net/org/info/
Please do look into breaking it! I'm sure there will be cases that will break this as well.
your lookarounds look a bit funny to me: you cant look behind inside a character class and why are you looking ahead? A look behind is imho more appropriate.
I think the following expression should work, although i didn't test it:
gsub("*((?<!#)[[:alnum:]~!#$%&+-=?,:/;._]*).com",'\\1',testset[10])
also note that lookbehinds must have a fixed length, so no multipliers are allowed
I have written this regex that works, but honestly, it’s like 75% guesswork.
The goal is this: I have lots of imports in Xcode, like so:
#import <UIKit/UIKit.h>
#import "NSString+MultilineFontSize.h"
and I only want to return the categories that contain +. There are also lots of lines of code throughout the source which include + in other contexts.
Right now, this returns all of the proper lines throughout the Xcode project. But if there is one thing I’ve learned from googling and searching Stack Overflow for regex tutorials, it is that there are LOTS of different ways to do things. I’d love to see all of the different ways you guys can come up with that make it either more efficient or more bulletproof regarding potential spoofs or misses.
^\#import+.[\"]*+.(?:(?!\+).)*+.*[\"]
Thanks in advance for all of your help.
Update
Also I suppose I’ll accept the answer of whoever does this with the shortest string, without missing any possible spoofs. But again, thanks to everyone who participates in this learning experience.
Resources from answers
This is an awesome resource for practicing regex from Dan Rasmussen: RegExr
The first thing I notice is that your + characters are misplaced: t+. matches t one or more times, followed by a single character .. I'm assuming you wanted to match the end of import, followed by one or more of any character: import.+
Secondly, # doesn't need to be escaped.
Here's what I came up with: ^#import\s+(.*\+.*)$
\s+ matches one or more whitespace character, so you're guaranteed that the line actually starts with #import and not #importbutnotreally or anything else.
I'm not familiar with xcode syntax, but the following part of the expression, (.*\+.*), simply matches any string with a + character somewhere in it. This means invalid imports may be matched, but I'm working under the assumption your trying to match valid code. If not, this will need to be modified to validate the importer syntax as well.
P.S. To test your expression, try RegExr. You can hover over characters to check what they do.
sed 's:^#import \(.*[+].*\):\1:' FILE
will display
"NSString+MultilineFontSize.h"
for your sample.
Today I got an email from my boss saying to change the regex in our java script code that goes onto our client's website from
[a-zA-Z0-9]+[a-zA-Z0-9_\.\-]
to
[a-zA-Z0-9]+[a-zA-Z0-9_\-\.]
because one of our clients were complaining that it wasn't regex best practices and it's causing problems with their CMS and their DB.
Looking at those two regexes, It appears to me they match the exact same thing.
the . and the - are swapped at the end, but that shouldn't make a difference. Should it?
Am I missing something?
The developer from our client's company is really adamant about us changing it.
Can someone shed some light?
Thanks!
There is no functional difference.
If anything is having issues with that regex, then it is a non-standard/buggy implementation. I recommend finding out exactly what the problem is.
While I see no reason to change it, I see no reason not to change it, so do what you wish.
Tip: I'm guessing the regex is written wrong. If I know what it is supposed to mean, I would write it:
[a-zA-Z0-9]+[_\.\-]?
If you use a - in a character group, it goes last otherwise it denotes a range of characters, like A-Z. If you're escaping it, like you are, then it can be anywhere.
It's possible the CMS or other code they use un-escapes the regex, so in this case it will throw errors if the - isn't the last character in the group. I would say that having as few escaped characters in a regular expression as possible makes it easier to read, but that's from a personal perspective.
We have a large solution with many projects in it, and throughout the project in forms, messages, etc we have a reference to a company name. For years this company name has been the same, so it wasn't planned for it to change, but now it has.
The application is specific to one state in the US, so localizations/string resource files were never considered or used.
A quick Find All instances of the word pulled up 1309 lines, but we only need to change lines that actually end up being displayed to the user (button text, message text, etc).
Code can be refactored later to make it more readable when we have time to ensure nothing breaks, but for time being we're attempting to find all visible instances and replace them.
Is there any way to easily find these "instances"? Perhaps a type of Regex that can be used in the Find All functionality in Visual Studio to only pull out the word when it's wrapped inside quotes?
Before I go down the rabbit hole of trying to make my job easier and spending far more time than it would have taken to just go line by line, figured I would see if anyone has done something like this before and has a solution.
You can give this a try. (I hope your code is under source control!)
Foobar{[^"]*"([^"]*"[^"]*")*[^"]*}$
And replace with
NewFoobar\1
Explanation
Foobar the name you are searching for
[^"]*" a workaround for the missing non greedy modifier. [^"] means match anything but " that means this matches anything till the first ".
([^"]*"[^"]*")* To ensure that you are matching only inside quotes. This ensures that there are only complete sets of quotes following.
[^"]* ensures that there is no quote anymore till the end of the line $
{} the curly braces buts all this stuff following your companies name into a capturing group, you can refer to it using \1
The VS regex capability is quite stripped down. It perhaps represents 20% of what can be done with full-powered regular expressions. It won't be sufficient for your needs. For example, one way to solve this quote-delimited problem is to use non-greedy matching, which VS regex does not support.
If I were in your shoes, I would write a perl script or a C# assembly that runs outside of Visual Studio, and simply races through all files (having a particular file extension) and fixes everything. Then reload into Visual Studio, and you are done. Well, if all went well with the regex anway.
Ultimately what you really must watch out for is code like this:
Log.WriteLine("Hello " + m_CompanyName + " There");
In this case, regex will think that "m_CompanyName" appears between two quotes - but it is not what you meant. In this case you need even more sophistication, and I think you'll find the answer with a special .net regular expression extension.