Regular expression to remove multiple segments in file names - regex

I have several files to rename. I will rename them using a batch renamer software. It can remove patterns using regular expression. Here are two file name examples:
13 (a 3f kd) kjhucj 1 _ m (52 min) - jrhkfq 324
647 fjk3 d [63h hh] _ (jhgbh) abim (238 min)
I want to remove all characters after _\s but keep the \(\d+\smin\) part. I want them to be renamed to:
13 (a 3f kd) kjhucj 1 (52 min)
647 fjk3 d [63h hh] (238 min)
I tried _\s.*[^(\(\d+\smin\))] but got weird results. I don't know if this is even possible.


What you're really trying to do is remove everything between _\s and (, as well as everything after ). Alternatively, you want to keep everything up to _\s, and between ( and ). You should aim to define the delimiters of your capture groups rather than the contents most of the time:
(.+?)_\s.*(\(\d+ min\)).*
In this case, instead of removing, keep the contents of capture groups 1 and 2 only. Many tools will allow you to say something like \1\2 in the replacement pattern.
If your tool removes an entire match and you insist on doing it like that, you will have to use a two pass approach, since you have two disconnected regions to remove. This would require a tool that supports lookahead and lookbehind, as most of them do.
The first pass would use
_\s.*(?=\(\d+ min\))
The second pass would remove
(?<=\(\d+ min\)).*


Square brackets are special characters in regular expressions: when you put a group of characters between a [ and a ] they are interpreted as matching any one character in the collection. However, here you seem to want to match a real [ character. You can make the [ not special by putting a backslash in front of it, so try replacing the [ in your regular expressions with a \[ and see whether that helps.

Related

Regex Multiple rows [duplicate]

I'm trying to get the list of all digits preceding a hyphen in a given string (let's say in cell A1), using a Google Sheets regex formula :
=REGEXEXTRACT(A1, "\d-")
My problem is that it only returns the first match... how can I get all matches?
Example text:
"A1-Nutrition;A2-ActPhysiq;A2-BioMeta;A2-Patho-jour;A2-StgMrktg2;H2-Bioth2/EtudeCas;H2-Bioth2/Gemmo;H2-Bioth2/Oligo;H2-Bioth2/Opo;H2-Bioth2/Organo;H3-Endocrino;H3-Génétiq"
My formula returns 1-, whereas I want to get 1-2-2-2-2-2-2-2-2-2-3-3- (either as an array or concatenated text).
I know I could use a script or another function (like SPLIT) to achieve the desired result, but what I really want to know is how I could get a re2 regular expression to return such multiple matches in a "REGEX.*" Google Sheets formula.
Something like the "global - Don't return after first match" option on regex101.com
I've also tried removing the undesired text with REGEXREPLACE, with no success either (I couldn't get rid of other digits not preceding a hyphen).
Any help appreciated!
Thanks :)

You can actually do this in a single formula using regexreplace to surround all the values with a capture group instead of replacing the text:
=join("",REGEXEXTRACT(A1,REGEXREPLACE(A1,"(\d-)","($1)")))
basically what it does is surround all instances of the \d- with a "capture group" then using regex extract, it neatly returns all the captures. if you want to join it back into a single string you can just use join to pack it back into a single cell:

You may create your own custom function in the Script Editor:
function ExtractAllRegex(input, pattern,groupId) {
return [Array.from(input.matchAll(new RegExp(pattern,'g')), x=>x[groupId])];
}
Or, if you need to return all matches in a single cell joined with some separator:
function ExtractAllRegex(input, pattern,groupId,separator) {
return Array.from(input.matchAll(new RegExp(pattern,'g')), x=>x[groupId]).join(separator);
}
Then, just call it like =ExtractAllRegex(A1, "\d-", 0, ", ").
Description:
input - current cell value
pattern - regex pattern
groupId - Capturing group ID you want to extract
separator - text used to join the matched results.

Edit
I came up with more general solution:
=regexreplace(A1,"(.)?(\d-)|(.)","$2")
It replaces any text except the second group match (\d-) with just the second group $2.
"(.)?(\d-)|(.)"
1 2 3
Groups are in ()
---------------------------------------
"$2" -- means return the group number 2
Learn regular expressions: https://regexone.com
Try this formula:
=regexreplace(regexreplace(A1,"[^\-0-9]",""),"(\d-)|(.)","$1")
It will handle string like this:
"A1-Nutrition;A2-ActPhysiq;A2-BioM---eta;A2-PH3-Généti***566*9q"
with output:
1-2-2-2-3-

I wasn't able to get the accepted answer to work for my case. I'd like to do it that way, but needed a quick solution and went with the following:
Input:
1111 days, 123 hours 1234 minutes and 121 seconds
Expected output:
1111 123 1234 121
Formula:
=split(REGEXREPLACE(C26,"[a-z,]"," ")," ")

The shortest possible regex:
=regexreplace(A1,".?(\d-)|.", "$1")
Which returns 1-2-2-2-2-2-2-2-2-2-3-3- for "A1-Nutrition;A2-ActPhysiq;A2-BioMeta;A2-Patho-jour;A2-StgMrktg2;H2-Bioth2/EtudeCas;H2-Bioth2/Gemmo;H2-Bioth2/Oligo;H2-Bioth2/Opo;H2-Bioth2/Organo;H3-Endocrino;H3-Génétiq".
Explanation of regex:
.? -- optional character
(\d-) -- capture group 1 with a digit followed by a dash (specify (\d+-) multiple digits)
| -- logical or
. -- any character
the replacement "$1" uses just the capture group 1, and discards anything else
Learn more about regex: https://twiki.org/cgi-bin/view/Codev/TWikiPresentation2018x10x14Regex

This seems to work and I have tried to verify it.
The logic is
(1) Replace letter followed by hyphen with nothing
(2) Replace any digit not followed by a hyphen with nothing
(3) Replace everything which is not a digit or hyphen with nothing
=regexreplace(A1,"[a-zA-Z]-|[0-9][^-]|[a-zA-Z;/é]","")
Result
1-2-2-2-2-2-2-2-2-2-3-3-
Analysis
I had to step through these procedurally to convince myself that this was correct. According to this reference when there are alternatives separated by the pipe symbol, regex should match them in order left-to-right. The above formula doesn't work properly unless rule 1 comes first (otherwise it reduces all characters except a digit or hyphen to null before rule (1) can come into play and you get an extra hyphen from "Patho-jour").
Here are some examples of how I think it must deal with the text

The solution to capture groups with RegexReplace and then do the RegexExctract works here too, but there is a catch.
=join("",REGEXEXTRACT(A1,REGEXREPLACE(A1,"(\d-)","($1)")))
If the cell that you are trying to get the values has Special Characters like parentheses "(" or question mark "?" the solution provided won´t work.
In my case, I was trying to list all “variables text” contained in the cell. Those “variables text “ was wrote inside like that: “{example_name}”. But the full content of the cell had special characters making the regex formula do break. When I removed theses specials characters, then I could list all captured groups like the solution did.

There are two general ('Excel' / 'native' / non-Apps Script) solutions to return an array of regex matches in the style of REGEXEXTRACT:
Method 1)
insert a delimiter around matches, remove junk, and call SPLIT
Regexes work by iterating over the string from left to right, and 'consuming'. If we are careful to consume junk values, we can throw them away.
(This gets around the problem faced by the currently accepted solution, which is that as Carlos Eduardo Oliveira mentions, it will obviously fail if the corpus text contains special regex characters.)
First we pick a delimiter, which must not already exist in the text. The proper way to do this is to parse the text to temporarily replace our delimiter with a "temporary delimiter", like if we were going to use commas "," we'd first replace all existing commas with something like "<<QUOTED-COMMA>>" then un-replace them later. BUT, for simplicity's sake, we'll just grab a random character such as  from the private-use unicode blocks and use it as our special delimiter (note that it is 2 bytes... google spreadsheets might not count bytes in graphemes in a consistent way, but we'll be careful later).
=SPLIT(
LAMBDA(temp,
MID(temp, 1, LEN(temp)-LEN(""))
)(
REGEXREPLACE(
"xyzSixSpaces:[ ]123ThreeSpaces:[ ]aaaa 12345",".*?( |$)",
"$1"
)
),
""
)
We just use a lambda to define temp="match1match2match3", then use that to remove the last delimiter into "match1match2match3", then SPLIT it.
Taking COLUMNS of the result will prove that the correct result is returned, i.e. {" ", " ", " "}.
This is a particularly good function to turn into a Named Function, and call it something like REGEXGLOBALEXTRACT(text,regex) or REGEXALLEXTRACT(text,regex), e.g.:
=SPLIT(
LAMBDA(temp,
MID(temp, 1, LEN(temp)-LEN(""))
)(
REGEXREPLACE(
text,
".*?("&regex&"|$)",
"$1"
)
),
""
)
Method 2)
use recursion
With LAMBDA (i.e. lets you define a function like any other programming language), you can use some tricks from the well-studied lambda calculus and function programming: you have access to recursion. Defining a recursive function is confusing because there's no easy way for it to refer to itself, so you have to use a trick/convention:
trick for recursive functions: to actually define a function f which needs to refer to itself, instead define a function that takes a parameter of itself and returns the function you actually want; pass in this 'convention' to the Y-combinator to turn it into an actual recursive function
The plumbing which takes such a function work is called the Y-combinator. Here is a good article to understand it if you have some programming background.
For example to get the result of 5! (5 factorial, i.e. implement our own FACT(5)), we could define:
Named Function Y(f)=LAMBDA(f, (LAMBDA(x,x(x)))( LAMBDA(x, f(LAMBDA(y, x(x)(y)))) ) ) (this is the Y-combinator and is magic; you don't have to understand it to use it)
Named Function MY_FACTORIAL(n)=
Y(LAMBDA(self,
LAMBDA(n,
IF(n=0, 1, n*self(n-1))
)
))
result of MY_FACTORIAL(5): 120
The Y-combinator makes writing recursive functions look relatively easy, like an introduction to programming class. I'm using Named Functions for clarity, but you could just dump it all together at the expense of sanity...
=LAMBDA(Y,
Y(LAMBDA(self, LAMBDA(n, IF(n=0,1,n*self(n-1))) ))(5)
)(
LAMBDA(f, (LAMBDA(x,x(x)))( LAMBDA(x, f(LAMBDA(y, x(x)(y)))) ) )
)
How does this apply to the problem at hand? Well a recursive solution is as follows:
in pseudocode below, I use 'function' instead of LAMBDA, but it's the same thing:
// code to get around the fact that you can't have 0-length arrays
function emptyList() {
return {"ignore this value"}
}
function listToArray(myList) {
return OFFSET(myList,0,1)
}
function allMatches(text, regex) {
allMatchesHelper(emptyList(), text, regex)
}
function allMatchesHelper(resultsToReturn, text, regex) {
currentMatch = REGEXEXTRACT(...)
if (currentMatch succeeds) {
textWithoutMatch = SUBSTITUTE(text, currentMatch, "", 1)
return allMatches(
{resultsToReturn,currentMatch},
textWithoutMatch,
regex
)
} else {
return listToArray(resultsToReturn)
}
}
Unfortunately, the recursive approach is quadratic order of growth (because it's appending the results over and over to itself, while recreating the giant search string with smaller and smaller bites taken out of it, so 1+2+3+4+5+... = big^2, which can add up to a lot of time), so may be slow if you have many many matches. It's better to stay inside the regex engine for speed, since it's probably highly optimized.
You could of course avoid using Named Functions by doing temporary bindings with LAMBDA(varName, expr)(varValue) if you want to use varName in an expression. (You can define this pattern as a Named Function =cont(varValue) to invert the order of the parameters to keep code cleaner, or not.)
Whenever I use varName = varValue, write that instead.
to see if a match succeeds, use ISNA(...)
It would look something like:
Named Function allMatches(resultsToReturn, text, regex):
UNTESTED:
LAMBDA(helper,
OFFSET(
helper({"ignore"}, text, regex),
0,1)
)(
Y(LAMBDA(helperItself,
LAMBDA(results, partialText,
LAMBDA(currentMatch,
IF(ISNA(currentMatch),
results,
LAMBDA(textWithoutMatch,
helperItself({results,currentMatch}, textWithoutMatch)
)(
SUBSTITUTE(partialText, currentMatch, "", 1)
)
)
)(
REGEXEXTRACT(partialText, regex)
)
)
))
)

Regular expression for non-consecutive characters

I'm trying to create a regular expression that validates the following requirements:
Simultaneous use of Cyrillic and numbers is possible (without spaces and special characters)
Simultaneous use of Latin and numbers is possible (without spaces and special characters)
Simultaneous use of Cyrillic and Latin characters is not possible
The first letter must be capitalized, cannot be a number
Sequence length - from 2 to 16 digits inclusive
It is impossible to use 3 or more identical symbols in a row
I am using the following solution:
(?:([A-Z][A-Za-z0-9]{1,15}|[А-Я][А-ЯЁа-яё0-9]{1,15}))$
How do I change the regex to match the last requirement?
I use Google Sheets, in which it is impossible to use negative lookahead.
Sorry for my English.

I don't you can do this with a single regex without lookbehinds.
But there are workarounds for the "don't repeat same character 3 times" functionality.
The workarounds could be simpler if RE2 supported backreferences, but it does not. So the resulting rule will be longer.
You may define a column ValidNoThreeRepeats like this:
=
NOT(
OR(
AND(MID(A1;1 ;1)=MID(A1;2 ;1);MID(A1;2 ;1)=MID(A1;3 ;1));
AND(MID(A1;2 ;1)=MID(A1;3 ;1);MID(A1;3 ;1)=MID(A1;4 ;1));
AND(MID(A1;3 ;1)=MID(A1;4 ;1);MID(A1;4 ;1)=MID(A1;5 ;1));
AND(MID(A1;4 ;1)=MID(A1;5 ;1);MID(A1;5 ;1)=MID(A1;6 ;1));
AND(MID(A1;5 ;1)=MID(A1;6 ;1);MID(A1;6 ;1)=MID(A1;7 ;1));
AND(MID(A1;6 ;1)=MID(A1;7 ;1);MID(A1;7 ;1)=MID(A1;8 ;1));
AND(MID(A1;7 ;1)=MID(A1;8 ;1);MID(A1;8 ;1)=MID(A1;9 ;1));
AND(MID(A1;8 ;1)=MID(A1;9 ;1);MID(A1;9 ;1)=MID(A1;10;1));
AND(MID(A1;9 ;1)=MID(A1;10;1);MID(A1;10;1)=MID(A1;11;1));
AND(MID(A1;10;1)=MID(A1;11;1);MID(A1;11;1)=MID(A1;12;1));
AND(MID(A1;11;1)=MID(A1;12;1);MID(A1;12;1)=MID(A1;13;1));
AND(MID(A1;12;1)=MID(A1;13;1);MID(A1;13;1)=MID(A1;14;1));
AND(MID(A1;13;1)=MID(A1;14;1);MID(A1;14;1)=MID(A1;15;1))
)
)
Or in a compacted way like this:
=NOT(OR(AND(MID(A1;1 ;1)=MID(A1;2 ;1);MID(A1;2 ;1)=MID(A1;3 ;1));AND(MID(A1;2 ;1)=MID(A1;3 ;1);MID(A1;3 ;1)=MID(A1;4 ;1));AND(MID(A1;3 ;1)=MID(A1;4 ;1);MID(A1;4 ;1)=MID(A1;5 ;1));AND(MID(A1;4 ;1)=MID(A1;5 ;1);MID(A1;5 ;1)=MID(A1;6 ;1));AND(MID(A1;5 ;1)=MID(A1;6 ;1);MID(A1;6 ;1)=MID(A1;7 ;1));AND(MID(A1;6 ;1)=MID(A1;7 ;1);MID(A1;7 ;1)=MID(A1;8 ;1));AND(MID(A1;7 ;1)=MID(A1;8 ;1);MID(A1;8 ;1)=MID(A1;9 ;1));AND(MID(A1;8 ;1)=MID(A1;9 ;1);MID(A1;9 ;1)=MID(A1;10;1));AND(MID(A1;9 ;1)=MID(A1;10;1);MID(A1;10;1)=MID(A1;11;1));AND(MID(A1;10;1)=MID(A1;11;1);MID(A1;11;1)=MID(A1;12;1));AND(MID(A1;11;1)=MID(A1;12;1);MID(A1;12;1)=MID(A1;13;1));AND(MID(A1;12;1)=MID(A1;13;1);MID(A1;13;1)=MID(A1;14;1));AND(MID(A1;13;1)=MID(A1;14;1);MID(A1;14;1)=MID(A1;15;1))))
The idea is to have a rule that compares 1st, 2nd and 3rd character, then another rule that compares 2nd, 3rd, 4th, then another rule for 3rd, 4th, 5th, and so on and so forth. You join this rules with an OR, since if any of those match, it means that at some place some repetition exists. Finally, you negate the whole expresion with a NOT
Than you can check that both your regex and that column are valid.

Donno with which script language you're using
If's in PHP code form,I'd be using `Filter_var($param1, FILTER_VALIDATE..., FILTER_FLAG..)` if i were in your shoes .
It makes your way into both **validating** n **sanitizing** your snippet.
**PEACE**.

Regular expression: Delete everything between X and Y in filename

I have a lot of files with the beginning xxx-yy. Both xxx and yy may vary. Example:
356-01 Nielsen - Sovnen, Op. 18.mp3
Everything between "356-01" and ".mp3" must be deleted so the new filename is:
356-01.mp3
".mp3" also varies. The expression should cover ".flac" also.

Assuming xxx and yy are digits, you can do
s/(\d\d\d-\d\d).*(\..+$)/\1\2/
The \. at the end is a literal period and .+$ means every character up to the end, so it should get the extension because the .* before it is greedy.
The find and replace were written between the slashes and use capture groups.

My question was based on that I thought to have made ​​some in advance. It now turns out that it does not work and therefore I have to reformulate the task. I'm sorry.
I have a lot of files beginning with 01, 02, 03 and so on. It wil never exeece 99. Example:
01 Nielsen - Sovnen, Op. 18.mp3
A 3-digit number* must be added to the beginning of the new filename and everything between "01" and ".mp3" must be deleted so the new filename is:
356-01.mp3
".mp3" also varies. The expression should cover ".flac" also.
*) What 3-digit number you use in your response is not important as the command will be added as a line in a larger bash script that I edit manually before each use

Remove the first character of each line and append using Vim

I have a data file as follows.
1,14.23,1.71,2.43,15.6,127,2.8,3.06,.28,2.29,5.64,1.04,3.92,1065
1,13.2,1.78,2.14,11.2,100,2.65,2.76,.26,1.28,4.38,1.05,3.4,1050
1,13.16,2.36,2.67,18.6,101,2.8,3.24,.3,2.81,5.68,1.03,3.17,1185
1,14.37,1.95,2.5,16.8,113,3.85,3.49,.24,2.18,7.8,.86,3.45,1480
1,13.24,2.59,2.87,21,118,2.8,2.69,.39,1.82,4.32,1.04,2.93,735
Using vim, I want to reomve the 1's from each of the lines and append them to the end. The resultant file would look like this:
14.23,1.71,2.43,15.6,127,2.8,3.06,.28,2.29,5.64,1.04,3.92,1065,1
13.2,1.78,2.14,11.2,100,2.65,2.76,.26,1.28,4.38,1.05,3.4,1050,1
13.16,2.36,2.67,18.6,101,2.8,3.24,.3,2.81,5.68,1.03,3.17,1185,1
14.37,1.95,2.5,16.8,113,3.85,3.49,.24,2.18,7.8,.86,3.45,1480,1
13.24,2.59,2.87,21,118,2.8,2.69,.39,1.82,4.32,1.04,2.93,735,1
I was looking for an elegant way to do this.
Actually I tried it like
:%s/$/,/g
And then
:%s/$/^./g
But I could not make it to work.
EDIT : Well, actually I made one mistake in my question. In the data-file, the first character is not always 1, they are mixture of 1, 2 and 3. So, from all the answers from this questions, I came up with the solution --
:%s/^\([1-3]\),\(.*\)/\2,\1/g
and it is working now.

A regular expression that doesn't care which number, its digits, or separator you've used. That is, this would work for lines that have both 1 as their first number, or 114:
:%s/\([0-9]*\)\(.\)\(.*\)/\3\2\1/
Explanation:
:%s// - Substitute every line (%)
\(<something>\) - Extract and store to \n
[0-9]* - A number 0 or more times
. - Every char, in this case,
.* - Every char 0 or more times
\3\2\1 - Replace what is captured with \(\)
So: Cut up 1 , <the rest> to \1, \2 and \3 respectively, and reorder them.

This
:%s/^1,//
:%s/$/,1/
could be somewhat simpler to understand.

:%s/^1,\(.*\)/\1,1/
This will do the replacement on each line in the file. The \1 replaces everything captured by the (.*)

:%s/1,\(.*$\)/\1,1/gc
.........................

You could also solve this one using a macro. First, think about how to delete the 1, from the start of a line and append it to the end:
0 go the the start of the line
df, delete everything to and including the first ,
A,<ESC> append a comma to the end of the line
p paste the thing you deleted with df,
x delete the trailing comma
So, to sum it up, the following will convert a single line:
0df,A,<ESC>px
Now if you'd like to apply this set of modifications to all the lines, you will first need to record them:
qj start recording into the 'j' register
0df,A,<ESC>px convert a single line
j go to the next line
q stop recording
Finally, you can execute the macro anytime you want using #j, or convert your entire file with 99#j (using a higher number than 99 if you have more than 99 lines).
Here's the complete version:
qj0df,A,<ESC>pxjq99#j
This one might be easier to understand than the other solutions if you're not used to regular expressions!

A regex to match between delimiters except when there is a colon that is not between double quotes?

This one is a little bit complicated and I'm not sure if it can be done.
The regex need to match everything between a , (comma) or [] (square brackets).
It must not match if there is a :
And now the tricky part.
If the : is between " " it can match.
I managed to create a regex that does everything except the last.
(?<=[[,])[^:]+?(?=[],])
So this is what it needs to match.
[ ItemName:Data, More Data, With a number "as: " item name]
I'm going to keep testing. Lets see if someone solves it.

It sounds like you're trying to specify a language that's really to complicated to parse using only regular expressions. Here's a pattern that matches what you've described, but probably won't work perfectly. It doesn't use look behinds so you need to select the first match group to get the contents.
/[\[,](("[^"\]]*"|[^:\[])*?)[\]\,]/
/[\[,] # Opening bracket or comma.
(("[^"\]]*" # Anything not including the closing bracket, in quotes...
|[^:\[] # or not including the colon...
))*? # repeated any number of times.
[\]\,]/x # Closing bracket or comma.
An example usage in Python:
import re
pattern = re.compile(r"""[\[,](("[^"\]]*"|[^:\[])*?)[\]\,]""", re.DEBUG)
for match in pattern.finditer('[1 2 3] [4 5] [6 : 7], "8 : 9", '):
print match.group(1)
Producing output:
1 2 3
4 5
"8 : 9"

I have good experience in using (perl) regexps in practise, so let me share my experience. If you are handling complex cases like this it is almost always best to do it step by step, unless you are in special ciscumstances (for example speed of execution is crucial).
So in this case I woud simply do it in two steps. First explode the data to chunks, i.e. something like (depending on your language)
split(/[][,]/)
and than accept or remove individual parts. In this case just remove parts which match this expression
/^([^"]*:.*|.*:[^"])$/
i.e. parts which include semicolon not surrounded with parantheses.
Clearly this deos not solve all the cases like With a number "as: " : "item" name, but I agree with Jeremy, than if you are trying to implement complicated syntax language, than it might not be the right thing to just throw few regexpes on it without deeper analysis (i.e. answering what exactly it should accept in wierd cases like [ 1:1, 2":"2,3":":3,4":":":"4,5":":"5], ...) and using appropriate aprroach to solve it (recursive syntax parser)