I have a source data file which I have been using JREPL.BAT to perform some very simple search and replace operations quite successfully. I now need to expand on that to do 2 jobs.
1. remove all lines that start with the string "appX_formContent". This line contain a lot of html output also, it all needs to be deleted on that line.
2. remove all lines that start with "Hex Payload:" and the subsequent line that comes with it.
This is an example of the input data file which shows 2 records. The delimiter between each record is the row that contains "-----------------".
-----------------
Message Headers
JMSCorrelationID: 60bb7750-e9e2-11e9-98bb-42010a961307
JMSPriority: 4
JMSRedelivered: false
Message Properties
app_trackingId: 190990a2-d8d8-43eb-814a-36ceba7a9111
appX_formInstanceIdentifier: FRM389083
appX_formContent: {"data":{"C7d14a6eb-70e7-402d-9d6e-4efd01ba561c":"N","Y","test.</p>\n<p>test form data to be informed </p>\n<p>...............</p>\n<p><strong>Update</strong></p>\n<p><strong>years</strong>"<p>supervision</p>","<p>:true,"c9377ae2-901d-4461-929c-c76e26dc6183":false}}}
app_sourceSystemId: source
app_eventCode: FORM_OUTPUT
app_instigatingUserId: 66
JMSXGroupSeq: 0
Hex Payload:
25 50 44 46 2D 31 2E 35 0D 0A 34 20 30 20 6F 62 6A 0D 0A 3C 3C 2F 54
-----------------
Message Headers
JMSCorrelationID: 641a80d0-e9e2-11e9-98bb-42010a961307
JMSPriority: 4
JMSTimestamp: 2019 10 08 16:43:40
JMSRedelivered: false
Message Properties
app_trackingId: a3c2fe93-ef71-4611-9605-9858ff67a6e8
appX_formInstanceIdentifier: FRM388843
appX_formContent: {"data":{"C7d14a6eb-70e7-402d-9d6e-4efd01ba561c":"N","Y","test.</p>\n<p>test form data to be informed </p>\n<p>...............</p>\n<p><strong>Update</strong></p>\n<p><strong>years</strong>"<p>supervision</p>","<p>:true,"c9377ae2-901d-4461-929c-c76e26dc6183":false}}}
app_sourceSystemId: source
app_eventCode: FORM_OUTPUT
app_instigatingUserId: 433
JMSXGroupSeq: 0
Hex Payload:
25 50 44 46 2D 31 2E 35 0D 0A 34 20 30 20 6F 62 6A 0D 0A 3C 3C 2F
-----------------
This is the batch file that I use to call jrepl - very simple.
call jrepl ".*(?:appX_formContent: .*)" "" /m /f "inpu.txt" /o "output.txt"
I've only tried to remove the appX_formContent line with the regex but it isn't producing any output. I'm not good with regex so help appreciated.
Not sure how to handle the second task of deleting the Hex Payload: line.
Related
I am struggling for a few weeks with zlib inflate alghorithm.
I would like to decompress packets from popular game called Tibia. They have compressed it with zlib inflate alghorithms. But it seems something is changed. Can you check it, maybe you will spot something I am missing?
Packet compressed: - cannot be inflated directly (its raw huffman fixed coding - probably)
DA 22 A6 CB 10 99 5F AA 50 9C 9A AA 90 A8 90 05 B4 2F B5 44 41 C3 B1 28 D7 CA 50 53 8F CB B3 44 A1 3C 35 33 3D A3 58 C1 C8 44 CF C0 40 21 BF 4A 0F
Packet decompressed using Reverse Engineering:
B4 16 2D 00 59 6F 75 20 73 65 65 20 61 20 6A 61 63 6B 65 74 20 28 41 72 6D 3A 31 29 2E 0A 49 74 20 77 65 69 67 68 73 20 32 34 2E 30 30 20 6F 7A 2E
Decompressed packet compressed with zlib deflate using CyberChef deflate / php deflate functions:
db 22 a6 cb 10 99 5f aa 50 9c 9a aa 90 a8 90 95 98 9c 9d 5a a2 a0 e1 58 94 6b 65 a8 a9 c7 e5 59 a2 50 9e 9a 99 9e 51 ac 60 64 a2 67 60 a0 90 5f a5 07 00
Data matches in a few places, but generally, its different. Do you know what could be the case?
I attach a picture from RE an inflating function: Screenshot from IdaPro
Here are packets in correct order:
http://wklej.org/hash/6aee9e223f0/txt/ - inflated correctly
http://wklej.org/hash/bd371e7f510/txt/ - inflated correctly
http://wklej.org/hash/8f15935dc15/txt/ - inflated correctly
And here is the packet that cannot be inflated...
CA059BC6043619009FC9FFFFE831
Your packet that cannot be inflated is likely part of a longer stream of compressed data, with other packets preceding it and following it. You need to decompress all of them as a single stream for the decompression to succeed.
Your first example is a portion of a deflate stream that references data that preceded it. So it is part of a larger deflate stream. You need all of the compressed data that preceded that piece in order to decompress that piece. Your last example (CA05...) also references preceding data, so it too is part of a larger stream with compressed data that preceded it.
In Get_Targetluns, i cloned the ZFS volume and shared with targetGroups and get the serial number of the disk
My serial number is in the form of 69 71 6e 2e 32 30 31 30 2d 30 38 2e
6f 72 67 2e
In BSTR string i get the serial number in form of 69716e2e323031302d30382e6f72672e
Now i want to copy this BSTR value to the
rgDestinationLuns[0]->m_deviceIdDescriptor[0]->m_rgIdentifiers[0]->m_rgbIdentifier
When i do the mem copy and copy the string its not working, always getting Missing Disk Error
Even in the VSS trace am not getting the expected value in the
Am getting some thing like this
[ 1:34:09.466 P:03C0 T:0878 CORHWUTC(2805) HWDIAG] * PARAM OUT:
m_rgbIdentifier:
[ 1:34:09.466 P:03C0 T:0878 CORHWUTC(2971) HWDIAG]
* PARAM OUT: 30 31 34 30 42 30 41 41 30 30 35 35 37 36 30 32 0140B0AA00557602
How to assign the BSTR value to the m_rgbIdentifier ?
I am learning little sas book. Below is a code from book. and raw data. The issue is when I run it, the final data set keeps missing the record at end of line, i.e., it keeps missing 75 and 56, and label them as missing ("."). Could anyone point out where could possible be the problem? When I add spaces after 75 and 56 at line ends, the problem is gone.
DATA class;
INFILE 'c:\MyRawData\Scores.dat';
INPUT Score ##;
RUN;
PROC UNIVARIATE DATA = class;
VAR Score;
TITLE;
RUN;
Data in that file:
56 78 84 73 90 44 76 87 92 75
85 67 90 84 74 64 73 78 69 56
87 73 100 54 81 78 69 64 73 65
after run it shows more like
56 78 84 73 90 44 76 87 92 .
85 67 90 84 74 64 73 78 69 .
87 73 100 54 81 78 69 64 73 65
My suspicion is that you have something wrong with your end of lines; either you have a spurious character, or your end of line isn't correct in some fashion. Most likely you are using a windows file and you are running in Unix, so you have
75CRLF85
and since Unix uses only LF for line terminator, it sees "75CR" endofline "85", not "75" endofline "85" like it should.
In that case you can either do what you did - add a space, though that likely will still leave some 'blank' records in there - or use TERMSTR in your infile statement to tell SAS how to properly read the file in.
Otherwise, you may have some spurious end characters - for example, if you pasted this from the web, it's possible you have a non-breaking space that is not converted to a regular space.
You can find out by doing this:
data _null_;
infile 'c:\rawdata\myfile.dat';
input #;
put _infile_ $HEX60.;
run;
The 60 is 2x the length of the line. That tells you what SAS is seeing. What you should see:
3536203738203834203733203930203434203736203837203932203735
3835203637203930203834203734203634203733203738203639203536
383720373320313030203534203831203738203639203634203733203635
Digits in ASCII are 30+digit, so 35 is a 5, 36 is a 6, etc. Space is 20. The first line:
35|36|20|37|38|20|38|34|20|37|33|20| ...
so 5 6 space 7 8 space 3 8 space 7 3 space. If you see something else after the 37 35, then you know there is a problem. You might see any of the following:
0A = Line feed.
0D = Carriage return.
A0 = Nonbreaking (web) space.
There are lots of other things you could see, but those are the most likely to trip you up. Pasting from the web is often a problem.
i have a spreadsheed in calc. with some records. There is a column that contains the following information
Ecole Saint-Exupery
Rue Saint-Malo 24
67544 Paris
Well i need to have those lines divided into at least three columns
name: Ecole Saint-Exupery
street: Rue Saint-Malo 24
postal code and town 67544 Paris
Or even better - i have divided the postal code and town into two seperate columns!?
Question: is this possible? Can (or should) i do this in calc (open document-formate)?
Do i need to have to use a regex and perl or am i able to solve this issues without an regex?
Note - finally i need to transfer the data into MySQL-database...
I look forward to a tipp...
greetings
BTW: you can see all the things in a real world-live-demo: http://192.68.214.70/km/asps/schulsuche.asp?q=a&a=50&s=1750 - see the filed
Schulname
Straße
PLZ Ort
These field contains three things - the name, the street and the Postal Code and the town!
Question: can this be divided into parts!? If you copy and paste the information - and drop it to calc then you get all the information in only one cell. How to divide and seperate all those information into three cells or even four?
BTW - i tried to translate the information to hex-code - see the follwoing...:
Staatl. Realschule Grafenau
Rachelweg 20
94481 Grafenau
00000000: 53 74 61 61 74 6C 2E 20 52 65 61 6C 73 63 68 75
00000010: 6C 65 20 47 72 61 66 65 6E 61 75 20 0A 52 61 63
00000020: 68 65 6C 77 65 67 20 32 30 0A 39 34 34 38 31 20
00000030: 20 47 72 61 66 65 6E 61 75 20 20
but i do not know if this helps here!??
Can you help me to solve the problem. Do i need to have a regex!?
Many thanks in advance for any and all help!
You may not need a regex. You should be able to take the contents of the cell in question and split it up using the newline character that is present. I am not familiar with calc, but if there is a split() or explode() function that returns an array, then splitting on a newline will yield the 3 pieces you are looking for.
I have to convert the following code into Qt c++
So I need to understand what these lines to.
#key in this code is a pem key file contents by openssl
key = KEY+#key.public_key.to_der
so i think this is converting that key to der format and then combining it with KEY.
is it right?
whats does this do? to_sizet(key.size)
As you can see the function returns [num].pack('V'). But i dont know wht does it do? I mean [num].pavck('V'). what is it?
And whats does this mean. key.size is it the strlen of key?
def write_crx
print "write crx..." if #verbose
key = KEY+#key.public_key.to_der
File.open(#crx, 'wb') do |file|
file << MAGIC
file << EXT_VERSION
file << to_sizet(key.size)
file << to_sizet(#sig.size)
file << key
file << #sig
File.open(#zip, 'rb') do |zip|
file << zip.read
end
end
puts "done at \"#{#crx}\"" if #verbose
end
def to_sizet num
return [num].pack('V')
end
Well I have one more Question. Forgot to add last time.
what does this line do?
KEY = %w(30 81 9F 30 0D 06 09 2A 86 48 86 F7 0D 01 01 01 05 00 03 81 8D 00).map{|s| s.hex}.pack('C*')
so i think this is converting that key to der format and then combining it with KEY. is it right?
Yes.
whats does this do? to_sizet(key.size) As you can see the function returns [num].pack('V'). But i dont know wht does it do? I mean [num].pavck('V'). what is it?
Well, just have a look at what the documentation on Array#pack says. With the V modifier, it just converts the array to a binary representation of a 32 bit unsigned integer type.
And whats does this mean. key.size is it the strlen of key?
Again, the Ruby documentation helps. String#size is indeed the string length.
what does this line do?
KEY = %w(30 81 9F 30 0D 06 09 2A 86 48 86 F7 0D 01 01 01 05 00 03 81 8D 00)
.map{|s| s.hex}.pack('C*')
%(…) defines an array where each of its entries, separated by whitespace is treated as a separate string. %w(one two) is essentially a shorthand for ["one", "two"].
After that, we have to look up (again in the documentation) the meanings of Array#map and String#hex.
map simply applies an operation to each element of an array and creates a new array with the result. The operation is s.hex – that is, parse each of the strings as a hexadecimal number.
The result is once again packed, this time using the C* operation which, the documentation tells us, converts the numbers into their corresponding (unsigned) 8-bit character codes.
In summary: that line converts the hexadecimal values into a string of their respective characters.
Afterthought: you should really install Ruby and play a bit with the interactive Ruby console, irb. Keying in the above line already helps a lot:
$ irb
>> %w(30 81 9F 30 0D 06 09 2A 86 48 86 F7 0D 01 01 01 05 00 03 81 8D 00).map{|s| s.hex}
=> [48, 129, 159, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 129, 141, 0]
>> %w(30 81 9F 30 0D 06 09 2A 86 48 86 F7 0D 01 01 01 05 00 03 81 8D 00).map{|s| s.hex}.pack('C*')
=> "0\201\2370\r\006\t*\206H\206\367\r\001\001\001\005\000\003\201\215\000"