i have a variable that contains the following string (where each dot stands for a non-printable character):
.[?1h.=.81..
which is this in hex:
ESC [ ? 1 h ESC = CR 8 1 CR LF
1b 5b 3f 31 68 1b 3d 0d 38 31 0d 0a
What i want is to isolate the '81'. The number 81 can change, so it can be for example 100 and uses 3 bytes in the string then but the number is always between the two "0x0d".
So the goal is to isolate all bytes (which are always numbers in ascii) between the two "0x0d" and save them as an integer in another variable.
Is this possible with only using bash? Would it be possible to work with regex?
You can do it like this:
a=$'\033[?1h\033=\r81\r\n' # or a=$'\x1b[?1h\x1b=\r81\r\n'
[[ $a =~ $'\r'([0-9]+)$'\r' ]] && echo ${BASH_REMATCH[1]}
The $'...' will interpret escape sequences in a string like \r, \n, octal representation \033 or hex representation \x1b
A simple Regex would capture the required decimal characters in hex as follows:
0[dD](\s*3(\d))*\s*0[dD]
Group 2 captures the decimal value, which is the hex value - 30, so only the second character.
Unfortunately only the last group is captured. If you can restrict yourself to a certain number of maximal decimal places you can simply duplicate the term as in
0[dD](\s*3(\d))(\s*3(\d))?(\s*3(\d))?\s*0[dD]
and replace it by
\2\4\6
to get the decimal value.
Edit
If your input is not hex but an ordinary string, it would look as follows
\x0d(\d)*\x0d
or with manual repetition (here 3x):
\x0d(\d)(\d)?(\d)?\x0d
with the same replacement pattern
\1\2\3
Edit2
In sed it should work as follows:
sed -n "s/^.*\x0d(\d)(\d)?(\d)?\x0d.*$/\1\2\3/"
now with start and end padding ^.*matcher.*$, and replacement pattern. s/search/replace/
Related
Can someone help me with this regex?
I would like to extract either 1. or 2.
1.
(2624594000) 303 days, 18:32:20.00 <-- Timeticks
.1.3.6.1.4.1.14179.2.6.3.39. <-- OID
Hex-STRING: 54 4A 00 C8 73 70 <-- Hex-STRING (need "Hex-STRING" ifself too)
0 <--INTEGER
"NJTHAP027" <- STRING
OR
2.
Timeticks: (2624594000) 303 days, 18:32:20.00
OID: .1.3.6.1.4.1.14179.2.6.3.39
Hex-STRING: 54 4A 00 C8 73 70
INTEGER: 0
STRING: "NJTHAP027"
This filedname and value will return different data each time. (The data will be variable.)
I don't need to get the field names and only want to get the values in order from the top (multi value)
(?s)[^=]+\s=\s(?<value_v2c>([^=]+)-)
https://regex101.com/r/lsKeEM/2
-> I can't extract the last STRING: "NJTHAP027" at all!
The named group value_v2c is already a group, so you can omit the inner capture group.
Currently the - char should always be matched in the pattern, but you can either match it or assert the end of the string.
As you are using negated character classes and [^=]+ and \s, you can omit the inline modifier (?s) as both already match newlines.
To match the 2. variation, you can update the pattern to:
[^=]+\s=\s(?<value_v2c>[^=]+)(?:-|$)
Regex demo
To get the 1. version, you can match all before the colon as long as it is not Hex-String.
Then in the group optionally match it.
[^=]+\s=\s(?:(?!Hex-STRING:)[^:])*:?\s*(?<value_v2c>(?:Hex-STRING: )?[^=]+?)(?: -|$)
Regex demo
I have a hexadecimal string with space separator for each byte.
eg., A1 B2 C3 D4 E5 FF 00 11 22 33 44 ...
I would like to use a regex validator to verify the user input is correct or not?
How could I write the regular expression to achieve this goal?
Something like this:
^[A-F0-9]{2}( [A-F0-9]{2})*$
Explanation:
^ - anchor: string start
[A-F0-9]{2} - two symbols in either 0..9 or A..F range
( [A-F0-9]{2})* - followed by space and two 0..9 or A..F symbols zero or more times
$ - anchor: string end
If you allow a..f as valid hexadecimal symbols
^[A-Fa-f0-9]{2}( [A-Fa-f0-9]{2})*$
I would like to propose a solution based on DRY principle
(Don't Repeat Yourself).
Instead of writing the same pattern (as Dmitry proposed), you can:
Write the pattern for 2 hex digits as a capturing group - ([A-F0-9]{2}).
"Call" it again using (?1).
So the whole pattern can be ^([A-F0-9]{2})( (?1))*$.
There are also other variants of "calling" a capturing group, e.g.
(?-1) - call the preceding group or
(?&name) - call a named group.
For details see https://www.regular-expressions.info/subroutine.html
I have a source of data that was converted from an oracle database and loaded into a hadoop storage point. One of the columns was a BLOB and therefore had lots of control characters and unreadable/undetectable ascii characters outside of the available codeset. I am using Impala to write regex replace function to parse some of the unicode characters that the regex library cannot understand. I would like to remove the offending 2 character hex codes BEFORE I use the unhex query function so that I can do the rest of the regex parsing with a "clean" string.
Here's the code I've used so far, which doesn't quite work:
'[2-7]{1}([A-Fa-f]|[0-9]{1})'
I've determined that I only need to capture \u0020-\u007f - or represented in the two bit hex - 20-7f
If my string looks like this:
010A000000153020405C00000000143020405CBC000000F53320405C4C010000E12F204058540100002D01
I would like to be able to capture 2 characters at a time (e.g. 01,0A,00) evaluate whether or not that fits the acceptable range of 2 byte hex I mentioned above and return only what is acceptable.
The correct output should be:
30 20 40 5C 30 20 40 5C 33 20 40 5C 4C 2F 20 40 58 and 54
However, my expression finds the first acceptable number in my first range (5) and starts the capture from there which returns the position or indexing wrong for the rest of the string... and this is the return from my expression -
010A0000001**53**0**20****40****5C**000000001**43**0**20****40****5C**BC000000F**53****32**0**40****5C****4C**010000E1**2F****20****40****58****54**010000**2D**01
I just don't know how to evaluate only two characters at a time in a mixed-length string. And, if they don't fit the expression, iterate to the next two characters. But only in two character increments.
My example: https://regex101.com/r/BZL7t0/1
I have added a Positieve Lookbehind to it. Which starts at the beginning of the string and then matches 2 characters at the time. This ensures that the group you're matching always has groups of 2 characters before it.
Positieve Lookbehind:
(?<=^(..)*)
Updated regex:
(?<=^(..)*)([2-7]{1}[A-Fa-f0-9]{1})
Preview:
Regex101
I'm parsing a file using Python 2.7 and I'm trying to find all occurrences of the following pattern
ASCII 06 followed by any two characters in the range from ASCII 0 to ASCII 255
Naive try #1 - [chr(6)][chr(0)-chr(255][chr(0)-chr(255]
fails with a message that indicates the range cannot be strings.
I've tried several other combos - no success.
The record that I'm parinsg was read in
sF = open('D:\Scratch\xxxxx.01', 'r')
record = sF.read()
Any help will be gratefully appreciated.
Thanx,
Doug
Given you're using python2.7, your open().read() will return bytes. You can use the following bytes regex to match ASCII 06 followed by 2 [0-255] bytes:
reg = re.compile(b'\x06..')
Note that I used . here as any byte will satisfy 0 - 255 (except the newline character \n (\xa0))
If you wanted to also match the newline character you could do one of the following things:
# Uses the DOTALL flag to force `.` to match `\n`
reg = re.compile(b'\x06..', re.DOTALL)
# Makes a character class which matches all bytes
reg = re.compile(b'\x06[\x00-\xff]{2}')
I needed a utililty function earlier today to strip some data out of a file and wrote an appaling regular expresion to do it. The input was a file with lots of line with the format:
<address> <11 * ascii character value> <11 characters>
00C4F244 75 6C 74 73 3E 3C 43 75 72 72 65 ults><Curre
I wanted to strip out everything bar the 11 characters at the end and used the following expression:
"^[0-9A-F+]{8}[\\s]{2}[0-9A-F\\s]{34}"
This matched to the bits I didn't want which I then removed from the original string. I'd like to see how you'd do this but the particular areas I couldn't get working were:
1: having the regex engine return the characters I wanted rather than the characters I didn't and
2: finding a way of repeating the match on a single ascii value followed by the space (eg "75 " = [0-9A-F]{2}[\s]{1}?) and repeating that 11 times rather than grabbing 34 characters.
Looking at it again the easiest thing to do would be to match to the last 11 characters of each input line but this isn't very flexible and in the interests of learning regex I would like to see how you can match through from the start of the sequence.
Edit: Thanks guys, this is what I wanted:
"(?:^[0-9A-F]{8} )(?:[0-9A-F]{2} ){11} (.*)"
Wish I could turn more than one of you green.
As the file has a fixed format, you could use this regular expression to just match the last 11 characters.
^.{44}(.{11})
Last eleven is:
...........$
or:
.{11}$
Matching a hex byte + space and repeat eleven times:
([0-9A-Fa-f]{2} ){11}
1) ^[0-9A-F+]{8}[\s]{2}[0-9A-F\s]{34}(.*)
Parens are used for grouping with extraction. How you retrieve it depends on your language context, but now some sort of $1 is set to everything after the initial pattern.
2) ^[0-9A-F+]{8}[\s]{2}(?:[0-9A-F\s]){11}\s(.*)
(?:) is grouping without extraction. So (?:[0-9A-F\s]){11} considers the subpattern there as a unit and looks for it repeated 11 times.
I'm assuming PCRE here, by the way.
The address and ascii char value are all hex so:
^[0-9A-F\s]{42}
Matching the end of the line would be
.{11}$
To match only the end, you can use a positive look behind.
"(?<=(^[0-9A-F+]{8}[\\s]{2}[0-9A-F\\s]{34}))(.*?)$"
This would match any character until the end of the line, providing that it is preceded by the "look behind" expression.
(?<=....) defines a condition that must be met before matching is possible.
I am a bit short of time, but if you look on the net for any tutorial that contain the words "regex" and "lookbehind", you will find good stuff (if a regex tutorial covers look ahead/behind, it will usually be pretty complete and advanced).
Another advice is to get a regex training tool and play with it. Have a look at this excellent Regex designer.
If you're using Perl, you could also use unpack(), to get each element.
my #data;
open my $fh, '<', $filename or die;
for my $line(<$fh>){
my($address,#list) = unpack 'a8xx(a2x)11xa11', $line;
my $str = pop #list;
# unpack the hexadecimal bytes
my $data = join '', map { pack 'H2',$_ } #list;
die unless $data eq $str;
push #data, [$address,$data,$str];
}
close $fh;
I also went ahead and converted the 11 hexadecimal codes back into a string, using pack().