How to use regular expression to find keys in hash - regex

Question

I have 6mio hashes and need to count how many of these have keys that start with AA00, AB10 and how many of them have keys starting with with both strings.
For each hash I have done this:
if (exists $hash{AA00}) {
$AA00 +=1;
}
if (exists $hash{AB10}) {
$AB10 += 1;
}
if (exists $hash{AA00} and exists $hash{AA10}) {
$both += 1;
}
but then I count only the number of hashes that contains exactly AA00 or AB10 as keys, but I would also like to count hashes that contain, say AA001. Can I use regular expression for this?

Answer 1

I completely misunderstood your question. To find the number of hashes with keys matching a regex (as opposed to the number of keys matching a regex in a single hash), you can still use the grep approach I outlined in my earlier answer. This time, however, you need to loop through your hashes (I assume you're storing them in an array if you have 6 million of them) and run grep twice on each one:
#!/usr/bin/perl
use strict;
use warnings;
use feature 'say';
my #array = (
{ AA00 => 'foo' },
{ AB10 => 'bar' },
{ AA001 => 'foo' },
{ AA00 => 'foo', AB10 => 'bar' }
);
my ($hashes_with_aa00, $hashes_with_ab10, $hashes_with_both) = (0, 0, 0);
foreach my $hash (#array) {
my $aa_count = grep { /^AA00/ } keys %$hash;
my $ab_count = grep { /^AB10/ } keys %$hash;
$hashes_with_aa00++ if $aa_count;
$hashes_with_ab10++ if $ab_count;
$hashes_with_both++ if $aa_count and $ab_count;
}
say "AA00: $hashes_with_aa00";
say "AB10: $hashes_with_ab10";
say "Both: $hashes_with_both";
Output:
AA00: 3
AB10: 2
Both: 1
This works, but is pretty poor in terms of performance: grep loops through every element in the list of keys for each hash, and we're calling it twice per hash!
Since we don't care how many keys match in each hash, only whether there is a match, a better solution would be any from List::MoreUtils. any works much like grep but returns as soon as it finds a match. To use any instead of grep, change this:
foreach my $hash (#array) {
my $aa_count = grep { /^AA00/ } keys %$hash;
my $ab_count = grep { /^AB10/ } keys %$hash;
$hashes_with_aa00++ if $aa_count;
$hashes_with_ab10++ if $ab_count;
$hashes_with_both++ if $aa_count and $ab_count;
}
to this:
use List::MoreUtils 'any';
foreach my $hash (#array) {
my $aa_exists = any { /^AA00/ } keys %$hash;
my $ab_exists = any { /^AB10/ } keys %$hash;
$hashes_with_aa00++ if $aa_exists;
$hashes_with_ab10++ if $ab_exists;
$hashes_with_both++ if $aa_exists and $ab_exists;
}
Note that I changed the variable names to better reflect their meaning.
This is much better in terms of performance, but as Borodin notes in a comment on your question, you're losing the speed advantage of hashes by not accessing them with specific keys. You might want to change your data structure accordingly.
Original Answer: Counting keys that match a regex in a single hash
This is my original answer based on a misunderstanding of your question. I'm leaving it up because I think it could be useful for similar situations.
To count the number of keys that match a regex in a single hash, you can use grep:
my $aa_count = grep { /^AA00/ } keys %hash;
my $ab_count = grep { /^AB10/ } keys %hash;
my $both = $aa_count + $ab_count;
As HunterMcMillen points out in the comments, there's no need to search through the hash keys again to get the total count; in this case, you can simply add the two subtotals. You can get away with this because the two patterns you're searching for are mutually exclusive; in other words, you cannot have a key that both begins with AA00 and AB10.
In the more general case, it might be possible for a single key to match both patterns (thanks Borodin). In that case, you cannot simply add up the two subtotals. For example, if you wanted your keys to merely contain AA00 or AB10 anywhere in the string, not necessarily at the beginning, you would need to do something like this:
my $aa_count = grep { /AA00/ } keys %hash;
my $ab_count = grep { /AB10/ } keys %hash;
my $both = grep { /(?:AA00|AB10)/ } keys %hash;
Note that this calls grep multiple times, which means traversing the entire hash multiple times. This could be done more efficiently using a single for loop like FlyingFrog and Kenosis did.