What mistake did I do for this unexpected negative lookahead subpattern? - regex

Question

I am actually working with a .tsv database whose headers are full of meaningful things for me.
I thus wanted to rip them off from the header to something that I & others users (non proficient with relational databases, so we mostly use Excel in the end to organize data and process it) would be more able to handle with Excel, by breaking them up with tabs.
Example header:
>(name1)database-ID:database2-ID:value1:value2
(I know this seems strange to put values in an header but this is descriptive of parameters of the third value associated to the header, that we don't have to mess here)
output as:
name1\tdatabase-ID\tdatabase2-ID\tvalue1\tvalue2\n
I thus pasted my data (headers, one per line) in EmEditor (BOOST syntax) and came with this regex:
>\((.*)\)(.*?)\:(.*?)\:(.*?)\:(.*?)\n
with each capturing group being then separated from others by inserting tabs between each others. It works, with perfect matches, no problem.
But I became aware there were malformed lines that didn't respected the logic of the whole database, and I wanted to make an expression to separate them at once.
If I make it with wrong lines it would be:
>(name1)database-ID:database2-ID:value1-1:value1-2\n
>(name2)database-ID:database2-ID:value2-1:value2-2\n
>(name3)database-ID:database2-ID:value3-1value3-2\n
Last line is ill-formed because it lacks the : between both last values.
I want it to be matched by working around the original expression that recognizes well-formed lines.
I perfectly know that I could came with different solutions by slightly tweaking my first expression for eliminating the good lines and retrieving misformed one after but
I don't want a solution to my process, I just want to understand what I made not well there; so that I become more educated (and not just more tricky by being able to circumvent my mistakes that I can't resolve):
I tried a negation of the above mentioned expression:
([^(>\((.*)\)(.*?)\:(.*?)\:(.*?)\:(.*?)\n)])
That doesn't match with anything.
I tried a negative lookahead, but It will be extremely, painfully slow then will match every 0-length matches possible in the document:
(?!(^>\((.*)\)(.*?)\:(.*?)\:(.*?)\:(.*?)\n))
I thus added a group capture for a string of characters behind,
but it doesn't work either:
(?!(^>\((.*)\)(.*?)\:(.*?)\:(.*?)\:(.*?)\n))(^.*?)
So please explain me where I have been wrong with the negating group ([^whatever]) and the use of the negative lookahead?

Answer 1

So please explain me where I have been wrong with the negating group ([^whatever]) and the use of the negative lookahead?
Let's address the question first: What does [^(pattern)] do?
You seem to have a misunderstanding and expect it to:
Match everything except the subpattern pattern. (Negation)
What it actually does is to:
Match any character that aren't (, p, a, t, ... n, ).
Therefore, the pattern
([^(>\((.*)\)(.*?)\:(.*?)\:(.*?)\:(.*?)\n)])
... Matches a character that aren't (, >, (, ... \n, ).
As for the negative lookahead, you're simply doing it wrong. The anchor ^ is in the wrong position, therefore your assertion will fail to provide any useful help. It's also not what negative lookaheads are for altogether.
(?!(^>\((.*)\)(.*?)\:(.*?)\:(.*?)\:(.*?)\n))
I'll explain what this does:
(?! Open negative lookahead group: Assert the position does not match this pattern, without moving the pointer position.
( Capturing group. The use of capturing groups in negative lookaheads are useless, as the subpattern in negative lookahead groups never matches.
^ Assert position at start of string.
>\( Literal character sequence ">(".
(.*) Capturing group which matches as many characters as possible except newlines, then backtracks.
\) Literal character ")".
(.*?) Capturing group with reluctant zero-to-one match of any characters except newlines.
\: Literal character ":".
(.*?)\:(.*?)\:(.*?)
\n A new line.
) Closes capturing group.
) Closes negative lookahead group. When this assertion is finished, the pointer position is same as beginning, and thus the resulting match is zero-length.
Note that the anchor is nested within the negative lookahead group. It should be at the start:
^(?!(>\((.*)\)(.*?)\:(.*?)\:(.*?)\:(.*?)\n))
While this doesn't return anything useful, it explains what is wrong, since you don't need a solution. ;)
In case you are in need of a solution suddenly, please refer to this relevant answer of mine (I'm not adding anything else into the post):
Rails 3 - Precompiling all css, sass and scss files in a folder

Answer 2

You could do this simply through PCRE Verb (*SKIP)(*F). The below regex would match all the bad-lines.
(?:^>\([^()]*\):[^:]*:[^:]*:[^:]*:[^:\n]*$)(*SKIP)(*F)|^.+
DEMO

Answer 3

Based on what I have been reading from Unihedron;
This is what I came for in emEditor:
^(?!>\(([A-Za-z0-9_\'\-]*?)\)(([A-Za-z0-9_\'\-]*?)\:){3}([A-Za-z0-9_\'\-]*?)\n).*\n
>(name1)database-ID:database2-ID:value1-1:value1-2
(NOT MATCH)
>(name2)database-ID:database2-ID:value2-1:value2-2
(NOT MATCH)
>(name3)database-ID:database2-ID:value3-1value3-2
(MATCH)
>(name3)database-ID::database2-ID:value3-1:value3-2
(MATCH)
(the character class avoid discarding names including special characters without making it possible to have two subsequent ":".)
I also could achieve the same results with:
(?!^>\(([A-Za-z0-9_\'\-]*?)\)(([A-Za-z0-9_\'\-]*?)\:){3}([A-Za-z0-9_\'\-]*?)\n)^.*\n
So I guess that all along capturing groups were what was messing with my lookahead.
Now I acknowledge that Avinash Raj is more efficient with the (*SKIP)(*F)|^.+ pattern, just that I didn't know about those functions and I also wanted to understand my logic / syntax mistake. (Thanks to Unihedron for that)