Regex to select newlines in between quotation marks - regex

Question

I have a string in Ruby that resembles the following:
{
"a boolean": true,
"multiline": "
my
multiline
value
",
"a normal key": "a normal value"
}
I would like to match only the newline characters in the substring:
"
my
multiline
value
",
This is so that I can replace them with escaped newline characters. The aim here is to make the JSON easier to work with in the long run.

Answer 1

UPdate - These regex work as expected.
From #faissaloo - it seemed to fail however on my large JSON.
I ran this large string using both regex:
PCRE https://regex101.com/r/3jtqea/1
Ruby https://regex101.com/r/1HVCCC/1
They both work identically, and without flaw.
If you have any other concerns, please let me know.
I think Ruby supports the Perl like constructs.
If so, it can be done in a single global find and replace.
Like this:
Edit - Ruby doesn't do Backtracking Control Verbs (*SKIP)(*FAIL)
so, to do this in Ruby code, requires the regex to be more explicit.
So, with a slight modification of the pcre/perl regex, the Ruby equivalent is:
Ruby
Find
(?-m)((?!\A)\G|(?:(?>[^"]*"[^"\r\n]*"[^"]*))*")([^"\r\n]*)\K\r?\n(?=[^"]*")((?:[^"\r\n]*"(?:(?>[^"]*"[^"\r\n]*"))*[^"]*)?)
Replace
\\n\3
https://regex101.com/r/BaqjEE/1
https://rextester.com/NVFD38349
Explained ( but it's complex )
(?-m) # Non-multiline mode safety check
( # (1 start), Prefix. Capture for debug
(?! \A ) # Not BOS
\G # Test where last match left off
| # or,
(?: # Optionally align to next " ( only used once )
(?> [^"]* " [^"\r\n]* " [^"]* )
)*
" # A new quote to test
) # (1 end)
( [^"\r\n]* ) # (2), Line break Preamble. Capture for debug
\K # Exclude from the match (group 0) up to this point
\r? \n # Line break to escape
(?= [^"]* " ) # Validate we have " closure
( # (3 start), Optional end quote and alignment.
# To be written back.
(?:
[^"\r\n]* "
(?: # Optionally align to next "
(?> [^"]* " [^"\r\n]* " )
)*
[^"]*
)?
) # (3 end)
# Ruby Code:
#----------------------
# #ruby 2.3.1
#
# re = /(?-m)((?!\A)\G|(?:(?>[^"]*"[^"\r\n]*"[^"]*))*")([^"\r\n]*)\K\r?\n(?=[^"]*")((?:[^"\r\n]*"(?:(?>[^"]*"[^"\r\n]*"))*[^"]*)?)/
# str = '{
# "a boolean": true,
# "a boolean": true,
# "a boolean": true,
# "a boolean": true,
# "multiline": "
# my
# multiline
# value
# asdf"
# ,
#
# "a multiline boo
# lean": true,
# "a normal key": "a multiline
#
# value"
# }'
# subst = '\\n\3'
#
# result = str.gsub(re, subst)
#
# # Print the result of the substitution
# puts result
For Pcre/Perl
Find
(?:((?:(?>[^"]*"[^"\n]*"[^"]*))+(*SKIP)(*FAIL)|"|(?!^)\G)([^"\n]*)\K\n(?=[^"]*")((?:[^"\n]*")?))
Replace
\\n$3
https://regex101.com/r/06naae/1
Explained ( but it's complex )
Note if you're on a windows box where editors need CRLF breaks,
add a \r in front of the LF, like this \r\n.
(?:
( # (1 start), Prefix capture, for debug
(?:
(?> [^"]* " [^"\n]* " [^"]* )
)+
(*SKIP) (*FAIL) # Consume false positives, but ignore them
# (need this to align next ")
| # or,
" # A new quote to test
| # or,
(?! ^ ) # Not BOS
\G # Test where last match left off
) # (1 end)
( [^"\n]* ) # (2), Preamble capture, for debug
\K # Exclude from the match (group 0) up to this point
\n # Line break to escape
(?= [^"]* " ) # Validate we have " closure
( # (3 start), End quote, to be written back
(?: [^"\n]* " )?
) # (3 end)
)

Answer 2

I think this could help you. You capture the \n inside the string and then can replace it:
"[^"]*(\n)*",
Test it

Answer 3

Another option would be something like this:
string = '{
"a boolean": true,
"multiline": "my
multiline
value",
"a normal value"
}'
puts string.match(/"(\w+)(\n+\w*)+"/).to_s.gsub!("\n", '\n')
This matches the regex in your string and then substitutes the newlines with escaped newlines.

Answer 4

Late answer, but you can use a regex like:
'"(?=\n).*?"'
Matches:
"
my
multiline
value
",
Demo:
Regex Demo & Explanation

Answer 5

If your multiline strings don't include commas (right before the line breaks) then you could use that in json every line has to end with ,, {, or [ or the next line has to start with } or ]:
json_string.gsub(/(?<!,|\{|\[)\n(?!\s*[}\]])/, '\n')
If you have commas in your strings (or curly and square brackets) you can improve this approach by adding more details to the list of valid line endings:
valid_line_ends = %w(true, false, ", }, ], { [)
line_end_matcher = valid_line_ends.map(&Regexp.method(:escape)).join('|')
json_string.gsub(/(?<!#{line_end_matcher})\n(?!\s*[}\]])/, '\n')