changing several expressions in one line in perl - regex

I want to take a line containing several expressions of the same structure, containing 4 digit hexa numbers, and changing the number in that structure according to a hash table. I tried using this next peace of code:
while ($line =~ s/14'h([0-9,a-f][0-9,a-f][0-9,a-f][0-9,a-f])/14'h$hash_point->{$1}/g){};
Where $hash_point is a pointer to the hash table.
But it tells me that I try to use an undefined value, when I tried running the fallowing code:
while ($line =~ s/14'h([0-9,a-f][0-9,a-f][0-9,a-f][0-9,a-f])/14'h----/g){print $1," -> ",$hash_point->{$1},"\n";};
It changed all the wanted numbers to "----" but printed out the values only 2 times (there were much more changes).
Where is the problem?

This is what I used in the end:
$line =~ s/14'h([0-9a-f][0-9a-f][0-9a-f][0-9a-f])/"14'h".$hash_point->{$1}/ge;
and in order to account for numbers not in the hash I've added:
$line =~ s/14'h([0-9a-f][0-9a-f][0-9a-f][0-9a-f])/"14'h".((hash_point->{$1}) or ($1))/ge;
I also wanted to know what numbers don't appear at the hash:
$line =~ s/14'h([0-9a-f][0-9a-f][0-9a-f][0-9a-f])/"14'h".(($hash_point->{$1}) or (print "number $1 didn't change\n") &&($1))/ge;
and finaly, I wanted to be able to control whether the massage from the previous stage would be printed, I've added the use of $flag which in defined only if I want the massages to appear:
$line =~ s/14'h([0-9a-f][0-9a-f][0-9a-f][0-9a-f])/"14'h".(($hash_point->{$1}) or (((defined($flag)) && (print "number $1 didn't change\n")) or ($1)))/ge;

Your regexp seems to work well for me except when hexa number is not present in the hash.
I tried:
#!/usr/bin/perl
use 5.10.1;
use strict;
use warnings;
use Data::Dumper;
my $line = q!14'hab63xx14'hab88xx14'hab64xx14'hab65xx14'hcdef!;
my $hash_point = {
ab63 => 'ONE',
ab64 => 'TWO',
ab65 => 'THREE',
};
while ($line =~ s/14'h([0-9,a-f][0-9,a-f][0-9,a-f][0-9,a-f])/14'h$hash_point->{$1}/g){};
say $line;
This produces:
Use of uninitialized value in concatenation (.) or string at C:\tests\perl\test5.pl line 15.
Use of uninitialized value in concatenation (.) or string at C:\tests\perl\test5.pl line 15.
14'hONExx14'hxx14'hTWOxx14'hTHREExx14'h
The errors are for numbers ab88 and cdef that are not keys in the hash.

Just a small correction, but both of your regexes don't do what you think it does.
/[a-f,0-9]/
Matches any character from a to f, 0 to 9, and a comma. You are looking for
/[a-z0-9]/
Not that this is what is breaking your program (M42 probably got it right, but we can't be sure unless you show us the hash).
Also, apologies, not enough rep to actually answer to other posts.
EDIT:
Well, you go through a lot of hoops in that answer, so here's how I'd do it instead:
s/14'h\K(\p{AHex}{4})/if (defined($hash_point->{$1})) {
$hash_point->{$1};
} else {
say $1 if $flag;
$1;
}/ge
Mainly because chaining and's and &&'s and sosuch generally makes for fairly hard-to-understand code. All whitespace is optional, so squash it for the one-liner!

Related

Manipulating backreferences for substitution in perl

As a part of an attempt to replace scientific numbers with decimal numbers I want to save a backreference into a string variable, but it doesn't work.
My input file is:
,8E-6,
,-11.78E-16,
,-17e+7,
I then run the following:
open FILE, "+<C:/Perl/input.txt" or die $!;
open(OUTPUT, "+>C:/Perl/output.txt") or die;
while (my $lines = <FILE>){
$find = "(?:,)(-?)(0|[1-9][0-9]*)(\.)?([0-9]*)?([eE])([+\-]?)([0-9]+)(?:,)";
$noofzeroesbeforecomma = eval("$7-length($4)");
$replace = '"foo $noofzeroesbeforecomma bar"';
$lines =~ s/$find/$replace/eeg;
print (OUTPUT $lines);
}
close(FILE);
I get
foo bar
foo bar
foo bar
where I would have expected
foo 6 bar
foo 14 bar
foo 7 bar
$noofzeroesbeforecomma seems to be empty or non-existant.
Even with the following adjustment I get an empty result
$noofzeroesbeforecomma = $2;
Only inserting $2 directly in the replace string gives me something (which is then, unfortunately, not what I want).
Can anyone help?
I'm running Strawberry Perl (5.16.1.1-64bit) on a 64-bit Windows 7 machine, and quite inexperienced with Perl
Your main problem is not using
use strict;
use warnings;
warnings would have told you
Use of uninitialized value $7 in concatenation (.) or string at ...
Use of uninitialized value $4 in concatenation (.) or string at ...
I would recommend you try and find a module that can handle scientific notation, rather than trying to hack your own.
Your code, in a working order might look something like this. As you can see, I have put a q() around your eval string to avoid it being evaluated before $7 and $4 exists. I also removed the eval itself, since while double eval on an eval is somewhat excessive.
use strict;
use warnings;
while (my $lines = <DATA>) {
my $find="(?:,)(-?)(0|[1-9][0-9]*)(\.)?([0-9]*)?([eE])([+\-]?)([0-9]+)(?:,)";
my $noof = q|$7-length($4)|;
$lines =~ s/$find/$noof/eeg;
print $lines;
}
__DATA__
,8E-6,
,-11.78E-16,
,-17e+7,
Output:
6
14
7
As a side note, not using strict is asking for trouble. Doing it while using a variable name such as $noofzeroesbeforecomma is asking for twice the trouble, as it is rather easy to make typos.
This is not about backreferences but the original problem, transforming numbers from scientific notation. I'm sure there are some cases in which this fails:
#!/usr/bin/env perl
use strict;
use warnings;
use bignum;
for (<DATA>) {
next unless /([+-]?\d+(?:\.\d+)?)[Ee]([+-]\d+)/;
print $1 * 10 ** $2 . "\n";
}
__DATA__
,8E-6,
,-11.78E-16,
,-17e+7,
Output:
0.000008
-0.000000000000001178
-170000000
I suggest you use the Regexp::Common::number plugin for the Regexp::Common module which will find all real numbers for you and allow you to replace those that have an exponent marker
This code shows the idea. using the -keep option makes the module put each component into one of the $N variables. The exponent marker - e or E - is in $7, so the number can be transformed depending on whether this was present
use strict;
use warnings;
use Regexp::Common;
my $real_re = $RE{num}{real}{-keep};
while (<>) {
s/$real_re/ $7 ? sprintf '%.20f', $1 : $1 /eg;
print;
}
output
Given your example input, this code produces the following. The values can be tidied up further using additional code in the substitution
,0.00000800000000000000,
,-0.00000000000000117800,
,-170000000.00000000000000000000,
The problem is that Perl can handle all those types of expressions. And since the standard item of data in Perl is the string, you would only need to capture the expression to use it. So, take this expression:
/(-?\d+(?:.\d+)?[Ee][+-]?\d+)/
to extract it from the surrounding text and use sprintf to format it, like Borodin showed.
However, if it helps you to see a better case of what you tried to do, this works better
my ( $whole, $frac, $expon )
= $line =~ m/(?:,)-?(0|[1-9]\d*)(?:\.(\d*))?[eE]([+\-]?\d+)(?:,)/
;
my $num = $expon - length( $frac );
Why not capture the sign with the exponent anyway, if you're going to do arithmetic with it?
It's better to name your captures and eschew eval when it's not necessary.
The substitution--as is--doesn't make much sense.
Really, since neither the symbols or the digits can be case sensitive, just put a (?i) at the beginning, and avoid the E "character class" [Ee]:
/((?i)-?\d+(?:.\d+)?e[+-]?\d+)/

Regular expression statement inside a while loop only matching and printing one of several expected matches

I've been struggling with this for a while and I was wondering if there was something obvious I've missed.
As programming learning/practice, I'm trying to put together a simple script for calculating the components of a restriction enzyme digest mix. However, first I need to get a list of enzyme stock concentrations.
I pulled all the individual pages from the New England Biolabs enzyme page, and my goal with this current script is to pull out the name of the enzyme and the concentrations available from the company.
This example works with a local copy of EcoRI (link included at bottom of submission).
use warnings;
use strict;
open(FILE,'productR0101.asp');
my $line;
my $counter;
my $array1;
my $array2;
my $array3;
my $concentration;
my #array4;
$counter = 1;
while ($line = <FILE>) {
chomp($line);
if ($counter == 6 ){
$array1 = $line;
$counter++;
}
else{
$counter++;
}
if ($line =~ m/.{8}units.ml/g) {
(#array4) =$line =~ m/.{8}units.ml/g;
print #array4;
}
}
print "\n".$array1;
exit;
Every file has the enzyme name on the sixth line of the file, so I just pulled that whole line. However, the concentrations are in different locations, so my approach was to read in the file one line at a time, and match to the units/ml tag.
My thinking was that it should print out the match for each line, if there was one, every time the while loop runs, effectively resulting in a string of separate print statements.
This is where I get messed up. There are six different locations in this file with a units/ml tag: three for 20,000 and three for 100,000.
I was expecting six different results printed, but when I run this, only one 100,000 units/ml result is returned.
I've tried all sorts of fixes. I tried concatenating strings, I tried storing it as a string, I tried concatenating it onto another array that never gets touched by the (#array4) = $line =~ m/.{8}units.ml/g line, and it either breaks it or gives the same result.
And finally, I apologize for any weird conventions. I'm still learning Perl, and my first experience programming was with MATLAB.
Also, the $array1, $array2, etc. exist because I was trying to keep track of exactly what was getting put where; my intention is to clean it up once I get it functional.
So does anyone have any ideas about what I'm doing wrong?
EDIT: the data source is the source code to each individual enzyme page. For this example, if you view the page source you get the complete input file I gave to the script.
Are the 20,000 units/ml at the start of the line? Because in that case, .{8} would fail to match - the dot doesn't match newlines, and 20,000_ is only 7 characters.
We really need to see the data you are processing, but it looks like you are storing only the last occurrence of /units.ml/ in #array4 because you are reading the file line by line.
I will add to this answer if you supplement your question, but for now I need to know
What your data looks like
What the mysterious /.{8}/ is for
Are you aware that $array1, $array2, and $array3, are scalars, as well as being very bad names for variables?
For now, here is a rewrite of your code using idiomatic Perl, and the $. variable that evaluates to the line number of the file most recently read
use strict;
use warnings;
open my $file, '<', 'productR0101.asp' or die $!;
my $array1;
my #array4;
while (my $line = <$file>) {
chomp $line;
$array1 = $line if $. == 6;
if ($line =~ m/.{8}units.ml/) {
#array4 = $line =~ m/.{8}units.ml/g;
print "#array4\n";
}
}
print "\n".$array1;
I can't exactly reproduce the behavior you've reported of only getting one of the 100,000 units/ml results, as I'm not exactly sure what your input data is. However, I think the problem is with the regular expression not having any captures. You should put parenthesis around the part of the regex match that you want to be returned to #array4. So instead of this:
#array4 = $line =~ m/.{8}units.ml/g;
Try this:
#array4 = $line =~ m/(.{8})units.ml/g;
#array4 = $line =~ /(.{8})units.ml/;
EDIT:
You also don't want to use the m/ and /g modifiers.

Perl, match one pattern multiple times in the same line delimited by unknown characters

I've been able to find similar, but not identical questions to this one. How do I match one regex pattern multiple times in the same line delimited by unknown characters?
For example, say I want to match the pattern HEY. I'd want to recognize all of the following:
HEY
HEY HEY
HEYxjfkdsjfkajHEY
So I'd count 5 HEYs there. So here's my program, which works for everything but the last one:
open ( FH, $ARGV[0]);
while(<FH>)
{
foreach $w ( split )
{
if ($w =~ m/HEY/g)
{
$count++;
}
}
}
So my question is how do I replace that foreach loop so that I can recognize patterns delimited by weird characters in unknown configurations (like shown in the example above)?
EDIT:
Thanks for the great responses thus far. I just realized I need one other thing though, which I put in a comment below.
One question though: is there any way to save the matched term as well? So like in my case, is there any way to reference $w (say if the regex was more complicated, and I wanted to store it in a hash with the number of occurrences)
So if I was matching a real regex (say a sequence of alphanumeric characters) and wanted to save that in a hash.
One way is to capture all matches of the string and see how many you got. Like so:
open (FH, $ARGV[0]);
while(my $w = <FH>) {
my #matches = $w =~ m/(HEY)/g;
my $count = scalar(#matches);
print "$count\t$w\n";
}
EDIT:
Yes, there is! Just loop over all the matches, and use the capture variables to increment the count in a hash:
my %hash;
open (FH, $ARGV[0]);
while (my $w = <FH>) {
foreach ($w =~ /(HEY)/g) {
$hash{$1}++;
}
}
The problem is you really don't want to call split(). It splits things into words, and you'll note that your last line only has a single "word" (though you won't find it in the dictionary). A word is bounded by white-space and thus is just "everything but whitespace".
What you really want is to continue to do look through each line counting every HEY, starting where you left off each time. Which requires the /g at the end but to keep looking:
while(<>)
{
while (/HEY/g)
{
$count++;
}
}
print "$count\n";
There is, of course, more than one way to do it but this sticks close to your example. Other people will post other wonderful examples too. Learn from them all!
None of the above answers worked for my similar problem. $1 does not seem to change (perl 5.16.3) so $hash{$1}++ will just count the first match n times.
To get each match, the foreach needs a local variable assigned, which will then contain the match variable. Here's a little script that will match and print each occurrence of (number).
#!/usr/bin/perl -w
use strict;
use warnings FATAL=>'all';
my (%procs);
while (<>) {
foreach my $proc ($_ =~ m/\((\d+)\)/g) {
$procs{$proc}++;
}
}
print join("\n",keys %procs) . "\n";
I'm using it like this:
pstree -p | perl extract_numbers.pl | xargs -n 1 echo
(except with some relevant filters in that pipeline). Any pattern capture ought to work as well.

What regular expression do I use for replacing numbers with leading zeros?

I have a bunch of strings like this:
my $string1 = "xg0000";
my $string2 = "fx0015";
What do I do to increase the number in the string by 1 but also maintain the leading zeros to keep the length of the string the same.
I tried this:
$string =~ s/(\d+)/0 x length(int($1)) . ($1+1)/e;
It doesn't seem to work on all numbers. Is regex what I'm supposet to use to do this or is there a better way?
How about a little perl magic? The ++ operator will work even on strings, and 0000 will magically turn into 0001.
Now, we can't modify $1 since it is readonly, but we can use an intermediate variable.
use strict;
use warnings;
my $string = "xg0000";
$string =~ s/(\d+)/my $x=$1; ++$x/e;
Update:
I didn't think of this before, but it actually works without a regex:
C:\perl>perl -we "$str = 'xg0000'; print ++$str;"
xg0001
Still does not solve the problem DavidO pointed out, with 9999. You would have to decide what to do with those numbers. Perl has a rather interesting solution for it:
C:\perl>perl -we "$str = 'xg9999'; print ++$str;"
xh0000
You can do it with sprintf too, and use the length you compute from the number of digits that you capture:
use strict;
use warnings;
my $string = "xg00000";
foreach ( 0 .. 9 ) {
$string =~ s/([0-9]+)\z/
my $l = length $1;
sprintf "%0${l}d", $1 + 1;
/e;
print "$string\n";
}
This is a really bad task to solve with a regexp. Increasing a number can change an unlimited number of digits, and can in fact also change the number of non-zero digits! Unless you have sworn an oath to use only regexes for a year, use regex to extract the number and then sprintf "%06d" $x+1 to regenerate the new number with the desired width.

Perl Regex Problem!

I am reading a string from a file:
2343,0,1,0 ... 500 times ...3
Above is an example of $_ when it is read from a file. It is any number, followed by 500 comma separated 0's/1's then the number 3.
while(<FILE>){
my $string = $_;
chomp($string);
my $a = chop($string);
my $found;
if($string=~m/^[0-9]*\,((0,|1,){$i})/){
$found = $&.$a;
print OTH $found,"\n";
}
}
I am using chop to get the number 3 from the end of the string. Then matching the first number followed by $i occurences of 0, or 1. The problem I'm having is that chop is not working on the string for some reason. In the if statement when I try to concat the match and the chopped number all I get returned is the contents of $&.
I have also tried using my $a = substr $a,-1,1; to get the number 3 and this also hasn't worked.
The thing that's odd is that this code works in Eclipse on Windows, and when I put it onto a Linux server it won't work. Can anyone spot the silly mistake I'm making?
As a rule, I tend always to allow for unseen whitespace in my data. I find that it makes my code more robust expecting that somebody didn't see an extra space at the end of a line or string (as in writing to a log). So I think this would solve your problem:
my ( $a ) = $string =~ /(\S)\s*$/;
Of course, since you know you are looking for a number, it's better to be more precise:
my ( $a ) = $string =~ /(\d+)\s*$/;
Take care of the end of line char… I can not test here but I assume you just chop a newline. Try first to trim your string then chop it. See for example http://www.somacon.com/p114.php
Instead of trying to do it that way, why not use a regexp to pull out everything you need in one go?
my $x = "4123,0,1,0,1,4";
$x =~ /^[0-9]+,((?:0,|1,){4})([0-9]+)/;
print "$1\n$2\n";
Produces:
0,1,0,1,
4
Which is pretty much what you're looking for. Both sets of needed answers are in the match variables.
Note that I included ?: in the front of the 0,1, matching so that it didn't end up in the output match variables.
I'm really not sure what you are trying to achieve here but I've tried the code on Win32 and Solaris and it works. Are you sure $i is the correct number? Might be easier to use * or ?
use strict;
use warnings;
while(<DATA>){
my $string = $_;
chomp($string);
my $a = chop($string);
print "$string\n";
my $found;
if($string=~m/^[0-9]*\,((0,|1,)*)/){
$found = $&.$a;
print $found,"\n";
}
}
__DATA__
2343,0,1,0,0,1,1,0,0,0,1,1,0,0,0,1,1,0,0,0,1,1,0,0,0,1,1,0,0,0,1,1,0,0,0,1,1,0,0,0,1,1,0,3
I don't see much reason to use a regex in this case, just use split.
use strict;
use warnings;
use autodie; # open will now die on failure
my %data;
{
# limit the scope of $fh
open my $fh, '<', 'test.data';
while(<$fh>){
chomp;
s(\s+){}g; # remove all spaces
my($number,#bin) = split ',', $_;
# uncomment if you want to throw away the 3
# pop #bin if $bin[-1] == 3;
$data{$number} = \#bin;
}
close $fh;
}
If all you want is the 3
while(<$fh>){
# the .* forces it to look for the last set of numbers
my($last_number) = /.*([0-9]+)/;
}