Writing a bubble sort using Perl regular expressions - regex

I'm beginning to learn perl and I'm writing a simple bubble sort using regular expressions. However, I can't get it to sort properly (alphabetically, delimiting by whitespace). It just ends up returning the same string. Can someone help? I'm sure it's something really simple. Thanks:
#!/usr/bin/perl
use warnings;
use strict;
my $document=<<EOF;
This is the beginning of my text...#more text here;
EOF
my $continue = 1;
my $swaps = 0;
my $currentWordNumber = 0;
while($continue)
{
$document =~ m#^(\w+\s+){$currentWordNumber}#g;
if($document =~ m#\G(\w+)(\s+)(\w+)#)
{
if($3 lt $1)
{
$document =~ s#\G(\w+)(\s+)(\w+)#$3$2$1#;
$swaps++;
}
else
{
pos($document) = 0;
}
$currentWordNumber++;
}
else
{
$continue = 0 if ($swaps == 0);
$swaps = 0;
$currentWordNumber = 0;
}
}
print $document;
SOLVED: I figured out the problem. I wasn't taking into account punctuation after a word.

If you just want to sort all the words, you don't have to use regular expressions... Simply splitting up the text by newlines and white spaces should be much faster:
sub bsort {
my #x = #_;
for my $i (0..$#x) {
for my $j (0..$i) {
#x[$i, $j] = #x[$j, $i] if $x[$i] lt $x[$j];
}
}
return #x;
}
print join (" ", bsort(split(/\s+/, $document)));

Related

How to remove newlines inside csv cells using regex/terminal tools?

I have a csv file where some of the cells have newline character inside. For example:
id,name
01,"this is
with newline"
02,no newline
I want to remove all the newline characters inside cells.
How to do it with regex or with other terminal tools generically without knowing number of columns in advance?
This is actually a harder problem than it looks, and in my opinion, means that regex isn't the right solution. Because you're dealing with quoting/escaped strings, spanning multiple 'lines' you end up with a complicated and difficult to read regex. (It's not impossible, it's just messy).
I would suggest instead - use a parser. Perl has one in Text::CSV and it goes a bit like this:
#!/usr/bin/env perl
use strict;
use warnings;
use Text::CSV;
my $csv = Text::CSV->new( { binary => 1, eol => "\n" } );
while ( my $row = $csv->getline( \*ARGV ) ) {
s/\n/ /g for #$row;
$csv->print( \*STDOUT, $row );
}
This will take files as piped in/specified on command line - that's what \*ARGV does - it's a special file handle that lets you do ... basically what sed does:
somecommand.sh | myscript.pl
myscript.pl filename_to_process
The ARGV filehandle doe either automagically. (You could explicitly open a file or use \*STDIN if you prefer)
I suspect that instead of removing the newline you actually want to replace it with a space. If your input file is as simple as it looks this should do it for you:
$ awk '{ORS=( (c+=gsub(/"/,"&"))%2 ? FS : RS )} 1' file
id,name
01,"this is with newline"
02,no newline
If you are using this xlsx2csv tool, it has this option:
-e, --escape Escape \r\n\t characters
Use it, and then replace \n as needed, like (if \n should be replaced by the empty string):
sed 's/\\n//g' filein.csv` > fileout.csv
In one pass:
PATH/TO/xlsx2csv.py -e filein.xlsx | sed 's/\\n//g' > fileout.csv
How to do it with regex or with other terminal tools generically without knowing number of columns in advance?
I don't think a regex is the most appropriate approach and might end up being quite complicated. Instead, I think a separate program to process the files might be easier to maintain in the long-term.
Since you're OK with any terminal tools, I've chosen python, and the code's below:
#!/usr/bin/python3 -B
import csv
import sys
with open(sys.argv[1]) as csvfile:
reader = csv.reader(csvfile)
for row in reader:
stripped = [col.replace('\n', ' ') for col in row]
print(','.join(stripped))
I think the code above is very straightforward and easy to understand, without a need for complicated regular expressions.
The input file here has the following contents:
id,name
01,"this is
with newline"
02,no newline
To prove it works, its output is reproduced below:
➜ ~ ./test.py input.csv
id,name
01,this is with newline
02,no newline
You could call the python script from some other program and feed filenames to it. You just need to add a minor update for the python program to write out files, if that's what you really need.
I've replaced the newlines with spaces to avoid a potentially unwanted concatenation (e.g. this iswith newline), but you can replace the newline with whatever you want, including the empty string ''.
I have written a method to remove the embedded new line inside the cell. The method below returns a java.util.List object that contains all rows in the CSV file
List<String> getAllRowsInCSVFileAsList(File selectedCSVFile){
FileReader fileReader = null;
BufferedReader reader = null;
List<String> values = new ArrayList<String>();
try{
fileReader = new FileReader(selectedCSVFile);
reader = new BufferedReader(fileReader);
String line = reader.readLine();
String previousLine = "";
//
boolean intendLineInCell = false;
while(line != null){
if(intendLineInCell){
if(line.indexOf("\"") != -1 && line.indexOf("\"") == line.lastIndexOf("\"")){
previousLine += line;
values.add(previousLine);
previousLine = "";
intendLineInCell = false;
} else if(line.indexOf("\"") != -1 && line.indexOf("\"") != line.lastIndexOf("\"")){
if(getTotalNumberOfCharacterSequenceOccurrenceInString("\"", line) % 2 == 0){
previousLine += line;
}else{
previousLine += line;
values.add(previousLine);
previousLine = "";
intendLineInCell = false;
}
} else{
previousLine += line;
}
}else{
if(line.indexOf("\"") == -1){
values.add(line);
}else if ((line.indexOf("\"") == line.lastIndexOf("\"")) && line.indexOf("\"") != -1){
intendLineInCell = true;
previousLine = line;
}else if(line.indexOf("\"") != line.lastIndexOf("\"") && line.indexOf("\"") != -1){
values.add(line);
}
}
line = reader.readLine();
}
}catch(IOException ie){
ie.printStackTrace();
}finally{
if(fileReader != null){
try {
fileReader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if(reader != null){
try {
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return values;
}
int getTotalNumberOfCharacterSequenceOccurrenceInString(String characterSequence, String text){
int count = 0;
while(text.indexOf(characterSequence) != -1){
text = text.replaceFirst(characterSequence, "");
count++;
}
return count;
}
Imagine you are creating a csv file with one row and five columns and in the 4th cell you have an embedded new line(enter inside the cell)
Your data will be look like below (actually we have only one row in csv but if you opened it in notepad it would look like 2 rows).
dinesh,kumar,24,"23
tambaram india",green
If there is a enter inside the cell could be like below
"23
tambaram india"
That cell starts with double quote(") and ends with double quote(").
Through using the double quote(") while reading the line if there is a double quote(") we can understand there is a embedded enter inside the cell.
The code concats the next line with that line and checks whether there is an end double quote(") or not. If there is, it adds a new row in the java.util.List object else it concats the next line and check it for end double quote(") and so on. Here I have explained for one cell, but the method also works if the row has a lot of cells with embedded enter.
Open the *csv file with notepadd++ and then press Ctrl+ H. Go to tab replace and enter to search box the "newline" and then write to replace the word you want to replace or let it empty if you want.

validating HTML fields in form using regex using perl

I have a couple of quick questions regarding using regex to validate some fields in a form. But I seem to be having some problems.
so here is the code
$userNameReg = "[a-zA-Z0-9_]+";
$passwordReg = "([a-zA-Z]*)([A-Z]+)([0-9]+)";
$emailReg = "[a-zA-Z0-9_]#[a-zA-Z]\.[a-zA-Z]{2,3}";
if ($onLoad !=1)
{
#controlValue = ($userName, $password, $phoneNumber, $email);
#regex = ($userNameReg, $passwordReg, "phoneNumber", $emailReg);
#validated;
for ($i=0; $i<4; $i++)
{
$retVal= validatecontrols ($controlValue[$i], $regex[$i]);
if ($retVal)
{
$count++;
}
if (!$retVal)
{
$validated[$i]="*"
}
}
sub validatecontrols
{
$ctrlVal = shift();
$regexVal = shift();
if ($ctrlVal =~ /$regexVal/)
{
return 1;
}
if ($ctrlVal !~ /$regexVal/)
{
return 0;
}
}
}
So what happens is that it still validates special characters, and I can't understand why. It does throw a flag if I enter a single special character but if its part of a word in the middle, beginning or end it validates.
Also please disregard the phone number part, because I haven't gotten to that part yet. I still have to create a regex that validates the phone number, digits only, first digit greater than 2.
Thank you all in advance for your help and insight.
Cheers
My guess is that you're missing start/end anchors. So [a-zA-Z0-9_]+ should be ^[a-zA-Z0-9_]+$. This way pattern will only match full string.
Also I strongly recommend you to enable use strict;. It can save you from a lot of mistype errors. Just add following to the beginning of the script:
use strict;
use warnings;
This will force perl to only allow defined variables. In most case you'll need to add my to first use of your variables (for example my $ctrlVal).
In validatecontrols you don't need second if statement. You can just return false like this:
sub validatecontrols
{
my $ctrlVal = shift();
my $regexVal = shift();
if ($ctrlVal =~ /$regexVal/)
{
return 1;
}
return 0;
}

My last regular expression won't work but i cannot figure out the reason why

I have two vectors, one which holds my regular expressions and one which holds the string in which will be checked against the regular expression, most of them work fine except for this one (shown below) the string is a correct string and matches the regular expression but it outputs incorrect instead of correct.
INPUT STRING
.C/IATA
CODE IS BELOW
std::string errorMessages [6][6] = {
{
"Correct Corparate Name\n",
},
{
"Incorrect Format for Corporate Name\n",
}
};
std::vector<std::string> el;
split(el,message,boost::is_any_of("\n"));
std::string a = ("");
for(int i = 0; i < el.size(); i++)
{
if(el[i].substr(0,3) == ".C/")
{
DCS_LOG_DEBUG("--------------- Validating .C/ ---------------");
output.push_back("\n--------------- Validating .C/ ---------------\n");
str = el[i].substr(3);
split(st,str,boost::is_any_of("/"));
for (int split_id = 0 ; split_id < splitMask.size() ; split_id++ )
{
boost::regex const string_matcher_id(splitMask[split_id]);
if(boost::regex_match(st[split_id],string_matcher_id))
{
a = errorMessages[0][split_id];
DCS_LOG_DEBUG("" << a )
}
else
{
a = errorMessages[1][split_id];
DCS_LOG_DEBUG("" << a)
}
output.push_back(a);
}
}
else
{
DCS_LOG_DEBUG("Do Nothing");
}
st[split_id] = "IATA"
splitMask[split_id] = "[a-zA-Z]{1,15}" <---
But it still outputs Incorrect format for corporate name
I cannot see why it prints incorrect when it should be correct can someone help me here please ?
Your regex and the surrounding logic is OK.
You need to extend your logging and to print the relevant part of splitMask and st right before the call to boost::regex_match to double check that the values are what you believe they are. Print them surrounded in some punctuation and also print the string length to be sure.
As you probably know, boost::regex_match only finds a match if the whole string is a match; therefore, if there is a non-printable character somewhere, or maybe a trailing space character, that will perfectly explain the result you have seen.

perl script to read content between marks

In the perl , how to read the contents between two marks. Source data like this
START_HEAD
ddd
END_HEAD
START_DATA
eee|234|ebf
qqq| |ff
END_DATA
--Generate at 2011:23:34
then I only want to get data between "START_DATA" and "END_DATA". How to do this ?
sub readFile(){
open(FILE, "<datasource.txt") or die "file is not found";
while(<FILE>){
if(/START_DATA/){
record(\*FILE);#start record;
}
}
}
sub record($){
my $fileHandle = $_[0];
while(<fileHandle>){
print $_."\n";
if(/END_DATA/) return ;
}
}
I write this code, it doesn't work. do you know why ?
Thanks
Thanks
You can use the range operator:
perl -ne 'print if /START_DATA/ .. /END_DATA/'
The output will include the *_DATA lines, too, but it should not be so hard to get rid of them.
Besides a few typos, your code is not too far off. Had you used
use strict;
use warnings;
You might have figured it out yourself. Here's what I found:
Don't use prototypes if you do not need them, or know what they do.
Normal sub declaration is sub my_function (prototype) {, but you can leave out the prototype and just use sub my_function {.
while (<fileHandle>) { is missing the $ sign to denote that it is
a variable (scalar) and not a global. Should be $fileHandle.
print $_."\n"; will add an extra newline. Just print; will do
what you expect.
if(/END_DATA/) return; is a syntax error. Brackets are not optional
in perl in this case. Unless you reverse the statement.
Use either:
return if (/END_DATA/);
or
if (/END_DATA/) { return }
Below is the cleaned up version. I commented out your open() while testing, so this would be a functional code example.
use strict;
use warnings;
readFile();
sub readFile {
#open(FILE, "<datasource.txt") or die "file is not found";
while(<DATA>) {
if(/START_DATA/) {
recordx(\*DATA); #start record;
}
}
}
sub recordx {
my $fileHandle = $_[0];
while(<$fileHandle>) {
print;
if (/END_DATA/) { return }
}
}
__DATA__
START_HEAD
ddd
END_HEAD
START_DATA
eee|234|ebf
qqq| |ff
END_DATA
--Generate at 2011:23:34
This is a pretty simple thing to do with regular expressions, just use the /s or /m (single line or multiple line) flags - /s allows the . operator to match newlines, so you can do /start_data(.+)end_data/is.

Why my perl script isn't finding bad indetation from my regex match

My work's coding standard uses this bracket indentation:
some declaration
{
stuff = other stuff;
};
control structure, function, etc()
{
more stuff;
for(some amount of time)
{
do something;
}
more and more stuff;
}
I'm writing a perl script to detect incorrect indentation. Here's what I have in the body of a while(<some-file-handle>):
# $prev holds the previous line in the file
# $current holds the current in the file
if($prev =~ /^(\t*)[^;]+$/ and $current =~ /^(?<=!$1\t)[\{\}].+$/) {
print "$file # line ${.}: Bracket indentation incorrect\n";
}
Here, I'm trying to match:
$prev: A line not ended with a semi-colon, followed by...
$current: A line not having the number of leading tabs+1 of the previous line.
This doesn't seem to match anything, at the moment.
the $prev variable needs some modification.
it should be something like \t* then .+ then not ending in semicolon
also, the $current should be like:
anything ending in ; or { or } not having the number of leading tabs+1 of the previous line.
EDIT
the perl code to try the $prev
#!/usr/bin/perl -l
open(FP,"example.cpp");
while(<FP>)
{
if($_ =~ /^(\t*)[^;]+$/) {
print "got the line: $_";
}
}
close(FP);
//example.cpp
for(int i = 0;i<10;i++)
{
//not this;
//but this
}
//output
got the line: {
got the line: //but this
got the line: }
it did not detect the line with the for loop ...
am i missing something...
i see a couple of problems...
your prev regex matches all lines which do not have a ; anywhere. which will break on lines like (for int x = 1; x < 10; x++)
if the indent of the opening { is incorrect, you will not detect that.
try this instead, it only cares if you have a ;{ (followed by any whitespace) at the end.
/^(\s*).*[^{;]\s*$/
now you should change your strategy so that if you see a line which does not end in { or ; you increment the indent counter.
if you see a line which ends in }; or } decrement your indent counter.
compare all lines against this
/^\t{$counter}[^\s]/
so...
$counter = 0;
if (!($curr =~ /^\t{$counter}[^\s]/)) {
# error detected
}
if ($curr =~ /[};]+/) {
$counter--;
} else if ($curr =~ /^(\s*).*[^{;]\s*$/) }
$counter++;
}
sorry for not styling my code according to your standards... :)
And you intend to only count tabs (not spaces) for indentation?
Writing this kind of checker is complicated. Just think about all the possible constructs that uses braces that should not change indentation:
s{some}{thing}g
qw{ a b c }
grep { defined } #a
print "This is just a { provided to confuse";
print <<END;
This {
$is = not $code
}
END
But anyway, if the issues above aren't important to you, consider whether the semi colon is important at all in your regex. After all, writing
while($ok)
{
sort { some_op($_) }
grep { check($_} }
my_func(
map { $_->[0] } #list
);
}
Should be possible.
Have you considered looking at Perltidy?
Perltidy is a Perl script that reformats Perl code into set standards. Granted, what you have isn't part of the Perl standard, but you can probably tweak the curly braces via the configuration file Perltidy uses. If all else fails, you can hack through the code. After all, Perltidy is just a Perl script.
I haven't really used it, but it might be worth looking into. Your problem is trying to locate all the various edge cases, and making sure you're handling them correctly. You can parse 100 programs to find that the 101st reveal problems in your formatter. Perltidy has been used by thousands of people on millions of lines of code. If there is an issue, it probably already has been found.