Matching newline in java regex without using dot [duplicate] - regex

I am currently trying to learn how to use regular expressions so please bear with my simple question. For example, say I have an input file containing a bunch of links separated by a newline:
www.foo.com/Archives/monkeys.htm
Description of Monkey's website.
www.foo.com/Archives/pigs.txt
Description of Pig's website.
www.foo.com/Archives/kitty.txt
Description of Kitty's website.
www.foo.com/Archives/apple.htm
Description of Apple's website.
If I wanted to get one website along with its description, this regex seems to work on a testing tool: .*www.*\\s.*Pig.*
However, when I try running it within my code it doesn't seem to work. Is this expression correct? I tried replacing "\s" with "\n" and it doesn't seem to work still.

The lines are probably separated by \r\n in your file. Both \r (carriage return) and \n (linefeed) are considered line-separator characters in Java regexes, and the . metacharacter won't match either of them. \s will match those characters, so it consumes the \r, but that leaves .* to match the \n, which fails. Your tester probably used just \n to separate the lines, which was consumed by \s.
If I'm right, changing the \s to \s+ or [\r\n]+ should get it to work. That's probably all you need to do in this case, but sometimes you have to match exactly one line separator, or at least keep track of how many you're matching. In that case you need a regex that matches exactly one of any of the three most common line separator types: \r\n (Windows/DOS), \n (Unix/Linus/OSX) and \r (older Macs). Either of these will do:
\r\n|[\r\n]
\r\n|\n|\r
Update: As of Java 8 we have another option, \R. It matches any line separator, including not just \r\n, but several others as defined by the Unicode standard. It's equivalent to this:
\r\n|[\n\x0B\x0C\r\u0085\u2028\u2029]
Here's how you might use it:
(?im)^.*www.*\R.*Pig.*$
The i option makes it case-insensitive, and the m puts it in multiline mode, allowing ^ and $ to match at line boundaries.

For future reference, one can also use the Pattern.DOTALL flag for "." to match even \r or \n.
Example:
Say the we are parsing a single string of http header lines like this (each line ended with \r\n)
HTTP/1.1 302 Found
Server: Apache-Coyote/1.1
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Frame-Options: SAMEORIGIN
Location: http://localhost:8080/blah.htm
Content-Length: 0
This pattern:
final static Pattern PATTERN_LOCATION = Pattern.compile(".*?Location\\: (.*?)\\r.*?", Pattern.DOTALL);
Can parse the location value using "matcher.group(1)".
The "." in the above pattern will match \r and \n, so the above pattern can actually parse the 'Location' from the http header lines, where there might be other headers before or after the target line (not that this is a recommended way to parse http headers).
Also, you can use "?s" inside the pattern to achieve the same effect.
If you are doing this, you might be better off using Matcher.find().

String str="I am a "+"\n Man of Peace"+"\t"+" .";
str=str.replaceAll("[\\s|\\t|\\r\\n]+"," ").trim();
System.out.println(str);
This above example works for tabSpaces, newLines, and normal spaces.
And I have used the trim method of java.lang.String to remove all the additional spaces in 'str'. I hope this helps you and other amazing people here.

try this
([^\r]+\r[^\r])+

Works for me:
import java.util.regex.Pattern;
import java.util.regex.Matcher;
public class Foo {
public static void main(String args[]) {
Pattern p = Pattern.compile(".*www.*\\s.*Pig.*");
String s = "www.foo.com/Archives/monkeys.htm\n"
+ "Description of Monkey's website.\n"
+ "\n"
+ "www.foo.com/Archives/pigs.txt\n"
+ "Description of Pig's website.\n"
+ "\n"
+ "www.foo.com/Archives/kitty.txt\n"
+ "Description of Kitty's website.\n"
+ "\n"
+ "www.foo.com/Archives/apple.htm\n"
+ "Description of Apple's website.\n";
Matcher m = p.matcher(s);
if (m.find()) {
System.out.println(m.group());
} else {
System.out.println("ERR: no match");
}
}
}
Perhaps the problem was with the way you were using the Pattern and Matcher objects?

This version matches newlines that may be either Windows (\r\n) or Unix (\n)
Pattern p = Pattern.compile("(www.*)((\r\n)|(\n))(.*Pig.*)");
String s = "www.foo.com/Archives/monkeys.htm\n"
+ "Description of Monkey's website.\n"
+ "\r\n"
+ "www.foo.com/Archives/pigs.txt\r\n"
+ "Description of Pig's website.\n"
+ "\n"
+ "www.foo.com/Archives/kitty.txt\n"
+ "Description of Kitty's website.\n"
+ "\n"
+ "www.foo.com/Archives/apple.htm\n"
+ "Description of Apple's website.\n";
Matcher m = p.matcher(s);
if (m.find()) {
System.out.println("found: "+m.group());
System.out.println("website: "+m.group(1));
System.out.println("description: "+m.group(5));
}
System.out.println("done");

Related

SCALA regex: Find matching URL rgex within a sentence

import java.util.regex._
object RegMatcher extends App {
val str="facebook.com"
val urlpattern="(http://|https://|file://|ftp://)?(www.)?([a-zA-Z0-9]+).[a-zA-Z0-9]*.[a-z]{3}.?([a-z]+)?"
var regex_list: Set[(String, String)] = Set()
val url=Pattern.compile(urlpattern)
var m=url.matcher(str)
if (m.find()) {
regex_list += (("date", m.group(0)))
println("match: " + m.group(0))
}
val str2="url is ftp://filezilla.com"
m=url.matcher(str2)
if (m.find()) {
regex_list += (("date", m.group(0)))
println("str 2 match: " + m.group(0))
}
}
This returns
match: facebook.com
str 2 match: url is ftp:
How do I manage the regex pattern so that both the strings are matched well.
What do the symbols actually mean in regex. I am very new to regex. Please help.
I read your regex as:
0 or 1 (? modifier) of the schemes (http://, https://, etc.)
followed by 0 or 1 instance of www.,
followed by 1 or more (+ modifier ) alphanumeric characters ,
followed by any character ( . is a regex special character, remember, standing for any one character),
followed by 0 or more (* modifier) alphanumerics,
followed by any character (. again)
followed by 3 lowercase letters ({3} being an exact count modifier)
followed by 0 or 1 of any character (.?)
followed by one or more lowecase letters.
If you plug your regex into regex101.com, you'll not only see a similar breakdown ( without any errors I might have made, though I think i nailed it), and you'll also have a chance to test various strings against it. Then, once you have your regexes working the way you want, you can bring them back to your script. It's a solid workflow for both learning regexes and developing an expression for a particular purpose.
If you drop your regex and your inputs into regex 101, you'll see why you're getting the output you see. But here's a hint: when you ask your regular expression to match "url is ftp://filezilla.com", nothing excludes "url is" from being part of the match. That's why you're not matching the scheme you want. Regex101 really is a great way to investigate this further.
The regex can be updated to
((ftp|https|http?):\/\/(?:www\.|(?!www))[a-zA-Z0-9][a-zA-Z0-9-]+[a-zA-Z0-9]\.[^\s]{2,}|www\.[a-zA-Z0-9][a-zA-Z0-9-]+[a-zA-Z0-9]\.[^\s]{2,})
This is all I needed.

AS3 Regex and the File.separator

I am on a Windows machine, and I am looking for a way to use Regex to count the number of occurrences of the File.separator characters in a path. Below is my code, and it outputs 0 every time.
var dummyPath:String = "C:" + File.separator + "A" + File.separator + "B.jpg";
var pattern:RegExp = new RegExp(File.separator,"g");
trace(dummyPath.match(pattern).length);
//Outputs 0
I'm not sure what else to do.
I wouldn't use a regex in a case like this, just because they're a lot more confusing to work with (and I think a lot more inefficient as well) than usual string operations, and you aren't doing anything here that's complicated enough to make up for the difference.
In that case, I would just go about it this way:
var dummyPath:String = "C:" + File.separator + "A" + File.separator + "B.jpg";
trace(dummyPath.split(File.separator).length - 1);
As for what you're running into though, remember that operating systems' file separators are generally either / or \. You're saying you're running this on Windows. That means you're passing "\" into the constructor for the regex. \ is used to begin escape sequences in regexes the same way it's used like that in strings.
So essentially you're not describing a regex that looks for instances of "\" on a Windows machine; you're describing a regex that starts an escape sequence and doesn't finish. So to use a regex in this case, you would need to escape \ with another \:
// This is technically untested, but the principle is the same.
var pattern:RegExp = new RegExp(File.separator.replace("\\", "\\\\"), "g");
Its not matching because the file separator you are using is a metacharacter.
The escape \.
The regex engine expects metachars, used as literals, to be escaped.
Try \\, which would be "\\\\" as a double quoted string.
If you run into a forward slash separator, just escape it too, does no harm.
So, concatenate the variable with an escape as a string Sep = "\\" + Sep; or something.

Regular expression extract filename from line content

I'm very new to regular expression. I want to extract the following string
"109_Admin_RegistrationResponse_20130103.txt"
from this file content, the contents is selected per line:
01-10-13 10:44AM 47 107_Admin_RegistrationDetail_20130111.txt
01-10-13 10:40AM 11 107_Admin_RegistrationResponse_20130111.txt
The regular expression should not pick the second line, only the first line should return a true.
Your Regex has a lot of different mistakes...
Your line does not start with your required filename but you put an ^ there
missing + in your character group [a-zA-Z], hence only able to match a single character
does not include _ in your character group, hence it won't match Admin_RegistrationResponse
missing \ and d{2} would match dd only.
As per M42's answer (which I left out), you also need to escape your dot . too, or it would match 123_abc_12345678atxt too (notice the a before txt)
Your regex should be
\d+_[a-zA-Z_]+_\d{4}\d{2}\d{2}\.txt$
which can be simplified as
\d+_[a-zA-Z_]+_\d{8}\.txt$
as \d{2}\d{2} really look redundant -- unless you want to do with capturing groups, then you would do:
\d+_[a-zA-Z_]+_(\d{4})(\d{2})(\d{2})\.txt$
Remove the anchors and escape the dot:
\d+[a-zA-Z_]+\d{8}\.txt
I'm a newbie in php but i think you can use explode() function in php or any equivalent in your language.
$string = "01-09-13 10:17AM 11 109_Admin_RegistrationResponse_20130103.txt";
$pieces = explode("_", $string);
$stringout = "";
foreach($i = 0;$i<count($pieces);i++){
$stringout = $stringout.$pieces[$i];
}

Regex: match whole line except first string and #comment lines

I tried (\s|\t).*[\b\w*\s\b], this one is almost ok but I want also except lines with #.
#Name Type Allowable values
#========================== ========= ========================================
_absolute-path-base-uri String -
add-xml-decl Boolean y/n, yes/no, t/f, true/false, 1/0
As #anubhava said in his answer, it looks you just need to check for # at the beginning of the line. The regex for that is simple, but the mechanics of applying the regex varies wildly, so it would help if we knew which regex flavor/tool you're using (e.g. PHP, .NET, Notepad++, EditPad Pro, etc.). Here's a JavaScript version:
/^[^#].*$/mg
Notice the modifiers: m ("multiline") allows ^ and $ to match at line boundaries, and g ("global") allows you to find all the matches, not just the first one.
Now let's look at your regex. [\b\w*\s\b] is a character class that matches a word character (\w), a whitespace character (\s), an asterisk (*), or a backspace (\b). In other words, both * and \b lose their special meanings when the appear in a character class.
\s matches any whitespace character including \t, so (\s|\t) is needlessly redundant, and may not be needed at all. What it's actually doing in your case is matching the newline before each matched line. There's no need for that when you can use ^ in multiline mode. If you want to allow for horizontal whitespace (i.e., spaces and tabs) before the #, you can do this:
/^(?![ \t]*#).*$/mg
(?![ \t]*#) is a negative lookahead; it means "from this position, it is impossible to match zero or more tabs or spaces followed by #". Coming right after the ^ line anchor as it does, "this position" means the beginning of a line.
Try this:
^[A-z0-9_-]+\s+(.+)$
Assuming your first string will consist of only letters, numbers, underscores or hyphens, the first part will match that. Then we match whitespace, and then capture the rest. However, this is all dependent on the regular expression engine being used. Is this using language support for regexes, a specific editor, or a certain library? Which one? There isn't a standard: each regex engine works slightly differently.
Try this:
^[^#].*?(\s|\t)(?<Group>.*)$
After a match is found, the Group group will contain your string.
I would use this regex. In English, this says "First character is not a pound sign (#), then non-white space to match the first 'word', then white space, then match the whole line.
^[^#]\S*\s+(.+)$
Can I suggest another approach though? It looks like there are tabs between each field in the text, so why not just read the text line-by-line and split by tab into an array?
Here is an example in C# (untested):
using(StreamReader sr = new StreamReader("C:\\Path\\to\\file.txt"))
{
string line = sr.ReadLine();
while(!sr.EndOfStream)
{
//skip the comment lines
if(line.StartsWith("#"))
continue;
string[] fields = line.Split(new string[] {"\t"}, StringSplitOptions.RemoveEmptyEntries);
//now fields[0] contains the Name field
//fields[1] contains the Type field
//fields[2] contains the Allowable Values field
line = sr.ReadLine();
}
}
Try this code in php:
<?php
$s="#Name Type Allowable values
#========================== ========= ========================================
_absolute-path-base-uri String -
add-xml-decl Boolean y/n, yes/no, t/f, true/false, 1/0 ";
$a = explode("\n", $s);
foreach($a as $str) {
preg_match('~^[^#].*$~', $str, $m);
var_dump($m);
}
?>
OUTPUT
array(0) {
}
array(0) {
}
array(1) {
[0]=>
string(79) "_absolute-path-base-uri String - "
}
array(1) {
[0]=>
string(77) "add-xml-decl Boolean y/n, yes/no, t/f, true/false, 1/0 "
}
Code is pretty simple, it just ignores matching # at the start of a line thus ingoring those lines completely.

Regex for quoted string with escaping quotes

How do I get the substring " It's big \"problem " using a regular expression?
s = ' function(){ return " It\'s big \"problem "; }';
/"(?:[^"\\]|\\.)*"/
Works in The Regex Coach and PCRE Workbench.
Example of test in JavaScript:
var s = ' function(){ return " Is big \\"problem\\", \\no? "; }';
var m = s.match(/"(?:[^"\\]|\\.)*"/);
if (m != null)
alert(m);
This one comes from nanorc.sample available in many linux distros. It is used for syntax highlighting of C style strings
\"(\\.|[^\"])*\"
As provided by ePharaoh, the answer is
/"([^"\\]*(\\.[^"\\]*)*)"/
To have the above apply to either single quoted or double quoted strings, use
/"([^"\\]*(\\.[^"\\]*)*)"|\'([^\'\\]*(\\.[^\'\\]*)*)\'/
Most of the solutions provided here use alternative repetition paths i.e. (A|B)*.
You may encounter stack overflows on large inputs since some pattern compiler implements this using recursion.
Java for instance: http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6337993
Something like this:
"(?:[^"\\]*(?:\\.)?)*", or the one provided by Guy Bedford will reduce the amount of parsing steps avoiding most stack overflows.
/(["\']).*?(?<!\\)(\\\\)*\1/is
should work with any quoted string
"(?:\\"|.)*?"
Alternating the \" and the . passes over escaped quotes while the lazy quantifier *? ensures that you don't go past the end of the quoted string. Works with .NET Framework RE classes
/"(?:[^"\\]++|\\.)*+"/
Taken straight from man perlre on a Linux system with Perl 5.22.0 installed.
As an optimization, this regex uses the 'posessive' form of both + and * to prevent backtracking, for it is known beforehand that a string without a closing quote wouldn't match in any case.
This one works perfect on PCRE and does not fall with StackOverflow.
"(.*?[^\\])??((\\\\)+)?+"
Explanation:
Every quoted string starts with Char: " ;
It may contain any number of any characters: .*? {Lazy match}; ending with non escape character [^\\];
Statement (2) is Lazy(!) optional because string can be empty(""). So: (.*?[^\\])??
Finally, every quoted string ends with Char("), but it can be preceded with even number of escape sign pairs (\\\\)+; and it is Greedy(!) optional: ((\\\\)+)?+ {Greedy matching}, bacause string can be empty or without ending pairs!
An option that has not been touched on before is:
Reverse the string.
Perform the matching on the reversed string.
Re-reverse the matched strings.
This has the added bonus of being able to correctly match escaped open tags.
Lets say you had the following string; String \"this "should" NOT match\" and "this \"should\" match"
Here, \"this "should" NOT match\" should not be matched and "should" should be.
On top of that this \"should\" match should be matched and \"should\" should not.
First an example.
// The input string.
const myString = 'String \\"this "should" NOT match\\" and "this \\"should\\" match"';
// The RegExp.
const regExp = new RegExp(
// Match close
'([\'"])(?!(?:[\\\\]{2})*[\\\\](?![\\\\]))' +
'((?:' +
// Match escaped close quote
'(?:\\1(?=(?:[\\\\]{2})*[\\\\](?![\\\\])))|' +
// Match everything thats not the close quote
'(?:(?!\\1).)' +
'){0,})' +
// Match open
'(\\1)(?!(?:[\\\\]{2})*[\\\\](?![\\\\]))',
'g'
);
// Reverse the matched strings.
matches = myString
// Reverse the string.
.split('').reverse().join('')
// '"hctam "\dluohs"\ siht" dna "\hctam TON "dluohs" siht"\ gnirtS'
// Match the quoted
.match(regExp)
// ['"hctam "\dluohs"\ siht"', '"dluohs"']
// Reverse the matches
.map(x => x.split('').reverse().join(''))
// ['"this \"should\" match"', '"should"']
// Re order the matches
.reverse();
// ['"should"', '"this \"should\" match"']
Okay, now to explain the RegExp.
This is the regexp can be easily broken into three pieces. As follows:
# Part 1
(['"]) # Match a closing quotation mark " or '
(?! # As long as it's not followed by
(?:[\\]{2})* # A pair of escape characters
[\\] # and a single escape
(?![\\]) # As long as that's not followed by an escape
)
# Part 2
((?: # Match inside the quotes
(?: # Match option 1:
\1 # Match the closing quote
(?= # As long as it's followed by
(?:\\\\)* # A pair of escape characters
\\ #
(?![\\]) # As long as that's not followed by an escape
) # and a single escape
)| # OR
(?: # Match option 2:
(?!\1). # Any character that isn't the closing quote
)
)*) # Match the group 0 or more times
# Part 3
(\1) # Match an open quotation mark that is the same as the closing one
(?! # As long as it's not followed by
(?:[\\]{2})* # A pair of escape characters
[\\] # and a single escape
(?![\\]) # As long as that's not followed by an escape
)
This is probably a lot clearer in image form: generated using Jex's Regulex
Image on github (JavaScript Regular Expression Visualizer.)
Sorry, I don't have a high enough reputation to include images, so, it's just a link for now.
Here is a gist of an example function using this concept that's a little more advanced: https://gist.github.com/scagood/bd99371c072d49a4fee29d193252f5fc#file-matchquotes-js
here is one that work with both " and ' and you easily add others at the start.
("|')(?:\\\1|[^\1])*?\1
it uses the backreference (\1) match exactley what is in the first group (" or ').
http://www.regular-expressions.info/backref.html
One has to remember that regexps aren't a silver bullet for everything string-y. Some stuff are simpler to do with a cursor and linear, manual, seeking. A CFL would do the trick pretty trivially, but there aren't many CFL implementations (afaik).
A more extensive version of https://stackoverflow.com/a/10786066/1794894
/"([^"\\]{50,}(\\.[^"\\]*)*)"|\'[^\'\\]{50,}(\\.[^\'\\]*)*\'|“[^”\\]{50,}(\\.[^“\\]*)*”/
This version also contains
Minimum quote length of 50
Extra type of quotes (open “ and close ”)
If it is searched from the beginning, maybe this can work?
\"((\\\")|[^\\])*\"
I faced a similar problem trying to remove quoted strings that may interfere with parsing of some files.
I ended up with a two-step solution that beats any convoluted regex you can come up with:
line = line.replace("\\\"","\'"); // Replace escaped quotes with something easier to handle
line = line.replaceAll("\"([^\"]*)\"","\"x\""); // Simple is beautiful
Easier to read and probably more efficient.
If your IDE is IntelliJ Idea, you can forget all these headaches and store your regex into a String variable and as you copy-paste it inside the double-quote it will automatically change to a regex acceptable format.
example in Java:
String s = "\"en_usa\":[^\\,\\}]+";
now you can use this variable in your regexp or anywhere.
(?<="|')(?:[^"\\]|\\.)*(?="|')
" It\'s big \"problem "
match result:
It\'s big \"problem
("|')(?:[^"\\]|\\.)*("|')
" It\'s big \"problem "
match result:
" It\'s big \"problem "
Messed around at regexpal and ended up with this regex: (Don't ask me how it works, I barely understand even tho I wrote it lol)
"(([^"\\]?(\\\\)?)|(\\")+)+"