PowerShell and RegEx formatting - regex
::EDIT::
After much goofing off, I was able to find a solution that appears to work in all cases... Consider the following:
$subject = '"LaunchPermission"=hex:01,00,14,80,64,00,00,00,74,00,00,00,14,00,00,00,30,00,00,00,02,00,1C,00,01,00,00,00,11,00,14,00,04,00,00,00,01,01,00,00,00,00,00,10,00,10,00,00,02,00,34,00,02,00,00,00,00,00,18,00,0B,00,00,00,01,02,00,00,00,00,00,0F,02,00,00,00,01,00,00,00,00,00,14,00,0B,00,00,00,01,01,00,00,00,00,00,01,00,00,00,00,01,02,00,00,00,00,00,05,20,00,00,00,20,02,00,00,01,02,00,00,00,00,00,05,20,00,00,00,20,02,00,00'
$result = $subject -creplace '(?ism)(.{1,76},)(.{1,75})', #'
$1\
$2\
'#
Write-Host $result
Note - Line 6 contains 2 spaces to get the indenting correctly.
This outputs exactly how I need! Thanks Fede for putting me on the right track!
Preface: I do realize there are other ways of achieving the end goal here, but within my current, larger scope, I need to format $subject in a specific way.
Good evening! Regex noob here. I'm attempting to find a way to format $subject in such a way that it is valid for dropping into a Windows .Reg file. At this point with the code below, I am able to return the first line exactly as I need it, but I'm struggling with trying to figure out how to create a second capture group that returns the values immediately after the first capture group.
Below is my current PowerShell code.
$subject = '"LaunchPermission"=hex:01,00,14,80,64,00,00,00,74,00,00,00,14,00,00,00,30,00,00,00,02,00,1C,00,01,00,00,00,11,00,14,00,04,00,00,00,01,01,00,00,00,00,00,10,00,10,00,00,02,00,34,00,02,00,00,00,00,00,18,00,0B,00,00,00,01,02,00,00,00,00,00,0F,02,00,00,00,01,00,00,00,00,00,14,00,0B,00,00,00,01,01,00,00,00,00,00,01,00,00,00,00,01,02,00,00,00,00,00,05,20,00,00,00,20,02,00,00,01,02,00,00,00,00,00,05,20,00,00,00,20,02,00,00'
$result = $subject -creplace '(?ism)(.{1,78},).*', '$1\'
Write-Host $result
This returns a $result of:
"LaunchPermission"=hex:01,00,14,80,64,00,00,00,74,00,00,00,14,00,00,00,30,00,\
From that point, I need to figure out how to create a second capture group so that it contains the remainder of the hex pairs that I can then apply additional formatting to.
The end goal is to have $result returned like this(for any similar value fed in via $subject):
"LaunchPermission"=hex:01,00,14,80,64,00,00,00,74,00,00,00,14,00,00,00,30,00,\
00,00,02,00,1c,00,01,00,00,00,11,00,14,00,04,00,00,00,01,01,00,00,00,00,00,\
10,00,10,00,00,02,00,34,00,02,00,00,00,00,00,14,00,0b,00,00,00,01,01,00,00,\
00,00,00,01,00,00,00,00,00,00,18,00,0b,00,00,00,01,02,00,00,00,00,00,0f,03,\
00,00,00,00,10,00,00,01,02,00,00,00,00,00,05,20,00,00,00,20,02,00,00,01,02,\
00,00,00,00,00,05,20,00,00,00,20,02,00,00
Any thoughts?
I'm not fully sure if this is what you want.
Using this regex:
(.{1,78},)(.{1,78})
You can check this working demo
Related
Regex to capture an URL
I've extracted an URL from a website in this string form:
#{href=http://download.company.net/file.exe}[0]
I can't figure out pattern how to get this part out of it: http://download.company.net/file.exe so I can use it as URL to download file.
From my point of view the logic would be, that I need to first match "http" as beggining of a string, wildcard inbetween and then match "}", but not include it in final output. So IDK ...[http]*\} (I know that this "syntax" of mine is totally wrong, but you get the idea)
Reason I dont want to include "exe" to pattern, is that file extension could be "msi" and I want it to be more universal. Also some good and comprehensive PS regex article would help me greatly (with inexperience in mind) - I really didnt find any "newbie friendly" or comprehensive enough to understand this topic.
You can either, use [regex]::match or -replace.
In the following example, I capture everything after href= that is not a starting curly bracket }:
'#{href=http://download.company.net/file.exe}[0]' -replace '#{href=([^}]+).*', '$1'
Output:
http://download.company.net/file.exe
I'd use -cmatch or -imatch as
if ($content -imatch '(?<=href=).*(?=})') {
$result = $matches[0]
} else {
$result = ''
}
In case of test data, it will return
http://download.company.net/file.exe
Regex gets more result then in text available
I have a really weird problem: i searching for URLs on a html site and want only a specific part of the url. In my test html page the link occurs only once, but instead of one result i get about 20... this is my regex im using: perl -ne 'm/http\:\/\myurl\.com\/somefile\.php.+\/afolder\/(.*)\.(rar|zip|tar|gz)/; print "$1.$2\n";' sample input would be something like this: <html><body>Somelinknme</body></html> which is a very easy example. so in real the link would apper on a normal website with content around... my result should be something like this: testfile.zip but instead i see this line very often... Is this a problem with the regex or with something else?
Yes, the regex is greedy.
Use an appropriate tool for HTML instead: HTML::LinkExtor or one of the link methods in WWW::Mechanize, then URI to extract a specific part.
use 5.010;
use WWW::Mechanize qw();
use URI qw();
use URI::QueryParam qw();
my $w = WWW::Mechanize->new;
$w->get('file:///tmp/so10549258.html');
for my $link ($w->links) {
my $u = URI->new($link->url);
# 'http://myurl.com/somefile.php?x=foo&y=bla&z=sdf&path=/foo/bar/afolder/testfile.zip&more=arguments&and=evenmore'
say $u->query_param('path');
# '/foo/bar/afolder/testfile.zip'
$u = URI->new($u->query_param('path'));
say (($u->path_segments)[-1]);
# 'testfile.zip'
}
Are there 20 lines following in the file after your link? Your problem is that the matching variables are not reseted. You match your link the first time, $1 and $2 get their values. In the following lines the regex is not matching, but $1 and $2 has still the old values, therefore you should print only if the regex matches and not every time. From perlre, see section Capture Groups NOTE: Failed matches in Perl do not reset the match variables, which makes it easier to write code that tests for a series of more specific cases and remembers the best match.
This should do the trick for your sample input & output. $Str = '<html><body>Somelinknme</body></html>'; #Matches = ($Str =~ m#path=.+/(\w+\.\w+)#g); print #Matches ;
How to Regex Multiple URLs From Same Variable In Perl
I'm trying to search a field in a database to extract URLs. Sometimes there will be more than 1 URL in a field and I would like to extract those in to separate variables (or an array).
I know my regex isn't going to cover all possibilities. As long as I flag on anything that starts with http and ends with a space I'm ok.
The problem I'm having is that my efforts either seem to get only 1 URL per record or they get only 1 the last letter from each URL. I've tried a couple different techniques based on solutions other have posted but I haven't found a solution that works for me.
Sample input line:
Testing http://marko.co http://tester.net Just about anything else you'd like.
Output goal
$var[0] = http://marko.co
$var[1] = http://tester.net
First try:
if ( $status =~ m/http:(\S)+/g ) {
print "$&\n";
}
Output:
http://marko.co
Second try:
#statusurls = ($status =~ m/http:(\S)+/g);
print "#statusurls\n";
Output:
o t
I'm new to regex, but since I'm using the same regex for each attempt, I don't understand why it's returning such different results.
Thanks for any help you can offer.
I've looked at these posts and either didn't find what I was looking for or didn't understand how to implement it:
This one seemed the most promising (and it's where I got the 2nd attempt from, but it didn't return the whole URL, just the letter: How can I store regex captures in an array in Perl?
This has some great stuff in it. I'm curious if I need to look at the URL as a word since it's bookended by spaces: Regex Group in Perl: how to capture elements into array from regex group that matches unknown number of/multiple/variable occurrences from a string?
This one offers similar suggestions as the first two. How can I store captures from a Perl regular expression into separate variables?
Solution:
#statusurls = ($status =~ m/(http:\S+)/g);
print "#statusurls\n";
Thanks!
I think that you need to capture more than just one character. Try this regex instead: m/http:(\S+)/g
How to remove a part of an URL with regexes?
How can I turn this: http://site.com/index.php?id=15 Into this?: http://site.com/index.php?id= Which RegEx(s) do I use? I've been trying to do this for a good 2 hours now and I've had no luck. I can't seem to take out the number(s) at the end, and sometimes there are letters in the end as well which give me problems. I am using Bing! instead of Google. My RegEx so far is this when I search something: $start = '<h3><a href="'; $end = '" onmousedown='; while ($result =~ m/$start(.*?)$end/g) What can I add in their to take out the letters and digits in the end and just leave it as an equal sign? Thank you.
Since you cannot parse [X]HTML properly with regular expressions, you should look for the minimum possible context that will get you the href you want. To the best of my knowledge, the one character that cannot be in a href is ". therefore /href="([^"]+)"/ Should yield a URL in $1. I would sanity check it for URL-ishness before extracting the id string you want, and then: s/\?id=\w+/id=/ But this has hack written all over it, because you can't parse HTML with regular expressions. So it will probably break the first time you demonstrate it to a customer. You should really check out proper Perl parsing: http://www.google.com/webhp?q=perl+html+parser
You asked for a regular expression solution but your problem is a bit ill-defined and regexes for HTML are only for stop-gap/one-off stuff or else you’re probably just hurting yourself.
Since I am really not positive what your actual need and HTML source look like this is a generic solution to taking a URL and spitting out all the links found on the page without query strings. Having id= is for all reasonable purposes/code equivalent to no id.
There are many ways, at least three or four of them good solutions, to do this in Perl. This is one that is often overlooked: libxml. Docs: XML::LibXML, URI, and URI::QueryParam (if you want better query manipulation).
use warnings;
use strict;
use URI;
use XML::LibXML;
my $source = shift || die "Give a URL!\n";
my $parser = XML::LibXML->new;
$parser->recover(1);
my $doc = $parser->load_html( location => $source );
for my $anchor ( $doc->findnodes('//a[#href]') )
{
my $uri = URI->new_abs( $anchor->getAttribute("href"), $source );
# commented out ideas.
# next unless $uri->host eq "TARGET HOST NAME";
# next unless $uri->path eq "TARGET PATH";
# Clear the query completely; id= might as well be nothing.
$uri->query(undef);
print $uri, $/;
}
It sounds like maybe you’re using Bing! for scraping. This kind of thing is against pretty much every search engine’s ToS. Don’t do it. They have APIs (well, Google does at least) if you register and get a dev token.
I'm not 100% sure what you are doing, but this is the problem:
while ($result =~ m/$start(.*?)$end/g)
What's the purpose of this loop? You're taking a scalar called $result and checking for a pattern match. How is $result changing?
Your original question was how to make this:
http://site.com/index.php?id=15
into this:
http://site.com/index.php?id=
That is, how do you remove the 15 (or another number) from the expression. The answer is pretty simple:
$url =~ s/=\d+$/=/;
That'll anchor your regular expression at the end of the URL replacing the ending digits with nothing.
If you're removing any string, it's a bit more complex:
$url =~ s/=[^=]+/=/;
You can't simply use \S+ because regular expressions are normally greedy. Therefore, you want to specify any series of non-equal sign characters preceded by an equal sign.
Now, as for the while loop, maybe you want an if statement instead...
if ($result =~ /$start(.*?)$end/g) {
print "Doing something if this matched\n";
}
else {
print "Doing something if there's no match\n";
}
And, I'm not sure what this means:
I am using Bing! instead of Google.
Are you trying to parse the input from Bing!? If so, please explain exactly what you're really trying to do. Maybe we know a better way of doing this. For example, if you're parsing the output of a search result, there might be an API that you can use.
How can I turn this: http://site.com/index.php?id=15 Into this?: http://site.com/index.php?id= I think this is the solution you are looking for #!/usr/bin/perl use strict; use warnings; my $url="http://site/index.php?id=15"; $url =~ s/(?<=id=).*//g; print $url; Output : http://site.com/index.php?id= as per your need anything after = sign will be omitted from the URL
Odd Perl Regex Behavior with Parens
I'm pulling in some Wikipedia markup and I'm wanting to match the URLs in relative (on Wikipedia) links. I don't want to match any URL containing a colon (not counting the protocol colon), to avoid special pages and the like, so I have the following code:
while ($body =~ m|<a href="(?<url>/wiki/[^:"]+)|gis) {
my $url = $+{url};
print "$url\n";
}
unfortunately, this code is not working quite as expected. Any URL that contains a parenthetical [i.e. /wiki/Eon_(geology)] is getting truncated prematurely just before the opening paren, so that URL would match as /wiki/Eon_. I've been looking at the code for a bit and I cannot figure out what I'm doing wrong. Can anyone provide some insight?
There isn't anything wrong in this code as it stands, so long as your Perl is new enough to support these RE features. Tested with Perl 5.10.1.
$body = <<"__ENDHTML__";
Body Blah blah
Body
__ENDHTML__
while ($body =~ m|<a href="(?<url>/wiki/[^:"]+)|gis) {
my $url = $+{url};
print "$url\n";
}
Are you using an old Perl?
You didn't anchor the RE to the end of the string. Put a " afterwards. While that is a problem, it isn't the problem he was trying to solve. The problem he was trying to solve was that there was nothing to match the method/hostname (http://en.wiki...) in the RE. Adding a .*? would help that, before the "(?"