A colleague has inserted duplicates for ~1200 entries into our database. They have sent me a text file containing both the originals and copies in alternating lines of CSV text. I've opened that up in VS Code with the goal of converting the lines representing duplicates into DELETE statements targeting our database. No line is truly identical to another—every two is a pair in which the data is the same other than the row ID.
I have found Stack Overflow entries for removing every other line when the line is empty, or when every other line is an exact copy of the previous line. I have not found an entry this scenario in which the lines have a difference. E.g. I tried using (.*)\n\1 w/ $1\n from another SO entry, which seems to target truly duplicate lines.
So how do I use VS Code to delete every other line regardless of content?
You can achieve this using Replace-All UI in regex mode.
Press command-F or control-F
Expand the arrow on the left of the Find display
Press the ".*" so that it's highlighted
Enter this for Find (top text field in the Find UI): (.*\n)(.*)\n (basically select two lines but save the contents of the first line in the regex system)
Enter this for Replace (following text field in the find UI): $1 (take the line saved from the Find regex and re-insert it)
Hit the Replace All button
Here's a similar SO question
Related
I am trying to validate a csv file using Apache-NiFi.
My CSV file has some defects.
id,name,address
1,sachith,{"Lane":"ABC.RTG.EED","No":"12"}
2,nalaka,{"Lane":"DEF",
"No":"23"}
3,muha,{"Lane":"GRF.FFF","No":"%$&%*^%"}
Here in second row,its been divided into two lines and third row has some special characters.
I want to ignore both the lines. For that I use \{("\w+":"\w+",)*[^%&*#]*\}, but this is not capturing row split error and new line.
I also used \{("\w+":"\w+",)*[^%&*#]*\}$, but it doesnt even get the right answer.
This is you might looking for: ^[0-9]+,[a-z]+,\{("\w+":"[\w\.]+","\w+":"[a-zA-Z0-9]+")\}$
Notepad++ has a Compare Plugin tool for comparing text files, which operates like this:
Launch Notepad++ and open the two files you wish to run a comparison
check on.
Click the “Plugins” menu,
Select “Compare” and click “Compare.”
The plugin will run a comparison check and display the two files side
by side, with any differences in the text highlighted.
This is a nice feature, and which I have used happily for some time. Now, I have been looking for an option to go further and select the highlighted differing lines (e.g. by deleting the non-highlighted ones), or vice versa: i.e. expunge the highlighted lines.
Is there a straightforward way to achieve this?
To substract two files in notepad++ (file1 - file2) you may follow this procedure:
Recommended: If possible, remove duplicates on both files, specially if the files are big. To do this: Edit => Line operations => Sort Lines Lexicographically Ascending (do it on both files)
Add ---------------------------- as a footer on file1 (add at least 10 dashes). This is the marker line that separates file1 content from file2.
Then copy the contents of file2 to the end of file1 (after the marker)
Control + H
Search: (?m-s)^(?:-{10,}+\R[\s\S]*+|(.*+)\R(?=(?:(?!^-{10,}$)-++|[^-]*+)*+^-{10,}+\R(?:^.*+\R)*?\1(?:\R|\z))) note: use case sensitivity according to your needs
Replace by: (leave empty)
Select Regular expression radio button
Replace All
You can modify the marker if It is possible that file1/file2 can have lines equal to the marker. In that case you will have to adapt the regular expression.
By the way, you could even record a macro to do all steps (add the marker, switch to file2, copy content to file1, apply the regex with a single button press.
Edited:
Changed the regex to add some improvements:
Speed related:
Avoid as much backtracking as possible
Avoid searching after the mark
Usability:
Dashes are allowed for the lines. But the separator is still ^-{10,}$
Works with other characters besides words
Speed comparison:
New method vs Old method
So basically 78ms vs 1.6seconds. So a nice improvement! That makes comparing Kilobyte-sized files possible.
Still you may want to use some dedicated program for comparing or substracting bigger files.
If the number of differences is not large, a quicker method might be just bookmarking each differing line using keyboard shortcuts. Starting from the beginning of the file, press Alt+Page Down to focus on the first difference, and then press Ctrl+F2 to bookmark it. Continue with alternatingly pressing Alt+Page Down and Ctrl+F2 until the last difference.
With all the differing lines bookmarked, you can use any of the operations under "Search -> Bookmarks" menu:
Cut Bookmarked Lines
Copy Bookmarked Lines
Paste to (Replace) Bookmarked Lines
Remove Bookmarked Lines
Remove Unmarked Lines
I have a dirty workaround for this. It saves some time compared to Control+C, Alt+Tab, Control+V; Control+C, Alt+Tab, Control+V; ... but It may not be worth on big files or if the differences for both files are big. For bigger files you may prefer using some other tool.
Typically this works best when comparing group of 'words' and does not work with content that is tabulated (like source code)
So the workaround is:
Optional: (depends on the content that's being compared) Sort both files (it will make the future comparison easier) To do this: Edit => Line operations => Sort Lines Lexicographically Ascending (do it on both files)
Compare files with the plugin
Choose one file and inspect the lines you want to keep. Add one tabulator before each of those lines. Remeber you can select several lines and press tab for tabulating them. Optionally, you may add tabulators to the lines you want to remove
Sort the file. The tabulated lines will come up first. So now you can copy-paste them (or copy-paste the untabulated ones)
move the files to a linux box and then execute diff command:
$ diff file1.txt file2.txt > file_diff.txt
I am struggling hard on this one.
I need to parse a portion of rtf1 document to rtf2 document. Here is the basic approach:
1) Open rtf1 in notepad and find the unique start point(line) and end point(line).
2) Copy portion from point start point to end point and insert it in rtf2 document.
I know how to insert the portion in rtf2 document but couldn't figure out how to extract the portion. The portion that needs to be copied is lengthy so I have to find a way to input start point and end point so I can use those two reference points to extract anything that falls in between.
Thank you in advance for your valuable input.
Zora
Find startPoint.*?endPoint using Regular expression -search mode with . matches newline enabled. Then copy selected text to clipoard.
.*? basically means "match any character until end point is found".
Remember to escape any special characters when defining the start- and endpoints.
|.......before.......|....portion-to-copy....|.....after..........|
If I need to perform such a task, I use simple method (in Notepad, Notepad++) - you can try it, too:
With text cursor at start point, you can press Ctrl+Shift+Home and then Delete. This deletes all content before your portion. (Do not save the file.)
With text cursor at end point, you can press Ctrl+Shift+End and then Delete. This deletes all content after your portion. (Do not save the file.)
Now you have only your portion. Press Ctrl+A (select all), then Ctrl+C to copy it into the clipboard. In SAS: Press Ctrl+Home, then press Ctrl+Shift+End. Then press Ctrl+C.
Paste your portion where you need.
Close your original document (used in steps 1-3) without saving.
I have a file containing (hundreds) of blocks of numbers like below;
This one is fine (16x20, correct number of rows and columns)
11111111111111111110
16666616666666661110
16111616111111162610
16111646111663132610
16162616261623132610
16162313261623132610
16162313261623132610
16162313261623132610
16162313261623132610
16162313261623132610
16162313261623132610
16162313261626132610
16166313661116632610
16111111111116132610
16666666666666136610
11111111111111111110
This one needs to be padded with trailing zeroes so it is (16x20)
111111111111111111
166616666666663661
166611111111111661
166666366663661661
113161111111161611
1316166666616161
1616162262616161
11616166112616161
16616166116616161
16616162262616161
16616166266616161
16616111161116161
1661666666666616111
1661666166163366661
1641666166166613661
1111111111111111111
I would like to pad them with zeroes so they are all like the first example. I'm aware of the regular expressions feature in notepad++ but am struggling to get it to work. I appreciate any help given.
You could do it via a macro.
First append a large number of zeroes to the end of each line using a macro.
Caret on the first entry
click record macro
press end
type out 20 zeroes
press down arrow
click stop recording
play the macro until all lines look like this
11111111111111111100000000000000000000000000000000000000000000
16661666666666366100000000000000000000000000000000000000000000
16661111111111166100000000000000000000000000000000000000000000
16666636666366166100000000000000000000000000000000000000000000
11316111111116161100000000000000000000000000000000000000000000
131616666661616100000000000000000000000000000000000000000000
161616226261616100000000000000000000000000000000000000000000
1161616611261616100000000000000000000000000000000000000000000
1661616611661616100000000000000000000000000000000000000000000
1661616226261616100000000000000000000000000000000000000000000
1661616626661616100000000000000000000000000000000000000000000
1661611116111616100000000000000000000000000000000000000000000
166166666666661611100000000000000000000000000000000000000000000
166166616616336666100000000000000000000000000000000000000000000
164166616616661366100000000000000000000000000000000000000000000
111111111111111111100000000000000000000000000000000000000000000
Then...
Caret on first line
click record
press home key
press the right arrow key 20 times
hold shift and press end key
press delete key
press down arrow
click stop recording
play the macro until all lines are processed
You could save the entire process as a single macro so its just a single click in the future.
I can give you a macro solution
go to the beginning of your text
select Macro/Start Recording
press end, press 0 16 times then press Home and down arrow key
select Macro/End Recording
You now have a macro to add sixteen zeros to the end of all lines.
Playback this macro on all lines.
You now have appended zeroes to all lines.
Pressing Alt key and using mouse select the required block(columns) of text you want and paste it into another empty notepad tab
help on column mode editing is there inside notepad ? / help contents menu
Good luck
You can use the plugin ConyEdit to do this.
With ConyEdit running in the background, follow these steps:
use the command line cc.aal 00000000000000000000 to append after lines with twenty zero character.
use the command line cc.gc 1/\d{20}/ to get the first column of regex match.
Looking to do this manualy and not progomaticly ?
Open Findreplace
Copy from the last to rhe first WITHOUT NUMBERS on a line so...
in this example
111111111111111111 <---from here
to here ---> 166616666666663661
166611111111111661
paste that into the fine ( yes your effecticly copying the return wich some applications allow you to manualy input others wont )
then in the replace box, type '0' then your return
Hit that magic replace all :D
This will then add a 0 every time it hits a new line, then add a new... new line....
edit : quickly reviewing another method a second to recover for alternate options :P give me 10
edit 2:
Ah ok somthing like this will work :P just tested it.
use [0-9] in the find replace. so if im looking for 123123123123 ( wich is 12 long ) and i need to buff i up to 20,
Your FIND must be in ()
so..
the find would be
([0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] )
and the replace is referd to as \1 no the regex, this was my mistake
\100000000
tested and confirmed !dont forget YOU NEED MATCH ALL on, WRAP off!
And so on for your other numbers, Not sure if you can loop this with macros nd stuff :P but hope it helps more than you have now
two good resources.
http://blog.creativeitp.com/posts-and-articles/editors/understanding-regex-with-notepad/comment-page-1/
http://regexpal.com/
base on OP's comment: you could try an editor called vim/gvim
open your file in vim, then type:
:%s/.*/\=printf("%-20s",getline("."))/|%s/ *$/\=substitute(submatch(0)," ","0","g")/
don't forget pressing <Enter> after the above typing.
then you will see the text has been changed into what you want.
of course vim macro can work as well, however, I feel command better... :)
I need to do quite a few regular expression search/replaces throughout hundreds and hundreds of static files. I'm looking to build an audit trail so I at least know what files were touched by what searches/replaces.
I can do my regular expression searches in Notepad++ and it gives me file names/paths and number of hits in each file. It also gives me the line #s which I don't really care that much about.
What I really want is a separate text file of the file names/paths. The # of hits in each file would be a nice addition, but really it's just a list of file names/paths that I'm after.
In Notepad++'s search results pane, I can do a right click and copy, but that includes all the line #s and code which is just too much noise, especially when you're getting hundreds of matches.
Anyone know how I can get these results to just the file name/paths? I'm after something like:
/about/foo.html
/about/bar.html
/faq/2012/awesome.html
/faq/2013/awesomer.html
/foo/bar/baz/wee.html
etc.
Then I can name that file regex_whatever_search.txt and at the top of it include the regex used for the search and replace. Below that, I've got my list of files it touched.
UPDATE What looks like the easiest thing to do (at least that I've found) is to just copy all the search results into a new text file and run the following regex:
^\tLine.+$
And replace that with an empty string. That'll give you just the file path and hit counts with a lot of empty space between each entry. Then run the following regex:
\s+\n
And replace with:
\n
That'll strip out all the unwanted empty space and you'll be left with a nice list.
maybe you need power of unix tools
assume you have GNUWin32 installed in c:\tools\gnuwin32
than if you have replace.bat file with that content:
#echo off
set BIN=c:\tools\gnuwin32\bin
set WHAT=%1
set TOWHAT=%2
set MASK=%3
rem Removing quotes
SET WHAT=###%WHAT%###
SET WHAT=%WHAT:"###=%
SET WHAT=%WHAT:###"=%
SET WHAT=%WHAT:###=%
SET TOWHAT=###%TOWHAT%###
SET TOWHAT=%TOWHAT:"###=%
SET TOWHAT=%TOWHAT:###"=%
SET TOWHAT=%TOWHAT:###=%
SET MASK=###%MASK%###
SET MASK=%MASK:"###=%
SET MASK=%MASK:###"=%
SET MASK=%MASK:###=%
echo %WHAT% replaces to %TOWHAT%
rem printing matching files
%BIN%\grep -r -c "%WHAT%" %MASK%
rem actual replace
%BIN%\find %MASK% -type f -exec %BIN%\sed -i "s/%WHAT%/%TOWHAT%/g" {} +
you can do regex replace in masked files recursively with output you required
replace "using System.Windows" "using Nothing" *.cs
The regulat expression I use for this kind of problem is
^\tLine.[0-9]*:.
And it works for me
This works well if you have Excel available and want to avoid using regular expressions:
Ctrl+A to select all the results
drag & drop the selected results to Excel
Create a Filter on the 1st row
Filter out the lines that have "(Blank)" on the 1st column
Select the remaining lines (i.e. the lines with the filenames) and copy/paste them to another sheet or any wanted destination
You could also Ctrl+A, Ctrl+C the search results, then use the Paste Option "Use Text Import Wizard" in Excel, say that the data is "Fixed width" and put one single break line after the 2nd character (to remove the two leading spaces in the filename during import), and use a filter to filter out the unwanted rows.