Currently I am working with a financial institution who get transaction through POS machine. The POS machine is from injenico company. I receive the HEXA message from the POS.
005B600002000002003020058020C010042100000000000200000000160021000200374520176553390700D20026011539608150000F31303030303030323130303030303030303030303030310000000000000000006303030303036.
How can I convert this to 8583 message. If I do normal conversion it gives a string:
`"‡\0S`\0\0\0\00 € À\0!\0\0\0\0\0\0\0\0\0\0!\0\07E eS9\a\0Ò\0&9`P\010000002100000000000001\0000006"`
I think it is encrypted string.
Can any one give me an idea about conversion of this ?
It is iso8583-1987-like financial message, very similar to Hypercom POS Data Handler (HPDH) message.
First two bytes are hex value of message size, then 5 bytes TPDU header, then iso 8583 message itself. Below is the data explanation as it is defined in HPDH specification.
FYI:
don't use real card data when you share dumps to public, it is
insecure.
when you tried to clean PIN block in DE052 (I guess) you lost one
hexadecimal character, because of this DE062 length was shifted. It
is fixed in the example parsing.
msg: # Hypercom POS Data Handler (HPDH) Message
TPDU: # Transport Protocol Data Unit
S01: "60" # ID // Transactions
S02: "0002" # Address, Destination
S03: "0000" # Address, Originator
MTI: "0200" # Message Type ID. // Financial Transaction Request
DE000: "3020058020C01004" # Primary bitmap // 3.4.11.22.24.25.35.41.42.52.62.
BM0: # Fields at Primary Bitmap
DE003: # PC
S01: "21" # Transaction Code. // Payment/Deposit
S02: "00" # Account, from. // Default
S03: "00" # Account, to. // Default
DE004: "000000020000" # Amount, transaction. // 20000
DE011: "000016" # STAN. // 16
DE022: # POS entry mode
PAD: "0"
S01: "02" # PAN entry mode // MS
S02: "1" # PIN entry capability // PIN
DE024: "0002" # Network International Identifier (NII)
DE025: "00" # POS condition code // Normal presentment
DE035: # Track 2 Data
len: "37"
val: "452017******0700D****6011539608150000F" # Track 2.
DE041: "10000002" # CATI.
DE042: "100000000000001" # CAIC.
DE052: "0000000000000000" # PIN block, encoded.
DE062: # Private data
len: "0006"
val:
v01: "000006" # Invoice/ECR reference number
Ingenico is just harware manufacturer, you must find out who wrote software for that POS. I guest that this is variation of ISO 8583 v1 (1987).
005B6000020000 Maybe some header for packet
0200 MTID
3020058020C01004 BITMAP (3, 4, 11, 22, 24, 25, 35, 41, 42, 52, 62)
210000 F03 Processing code in BCD
000000020000 F04 Amount in BCD
000016 F11 STAN in BCD
0021 F22 Point of service entry mode in BCD
0002 F24 ?
00 F25 POS condition code in BCD
374520176553390700D20026011539608150000F F35 Track2 in BCD
3130303030303032313030303030303030303030303031 F41 and F42 in ASCII
0000000000000000 F52 Pinblock (empty)
006303030303036. F62 Some private data
Related
I have a Python program which is accessing one of the devices on our Solar Power system. I can read the registers which are supposed to conform to the SunSpec conventions. I have been able to decode most of the values, but I'm stuck on decoding the TCP_Address and gateway which are sourced from these two registers:
TCP Address:
reg 22 value 49320 in HEX 0xc0a8
reg 23 value 64 in HEX 0x40
Gataway Address:
reg 24 value 49320 in HEX 0xc0a8
reg 25 value 1 in HEX 0x1
the documentation says that the format for these values is "uint32", which I interpret to mean unsigned 32 bit integer. The result of decoding should be something like 192.168.0.?.
Can anyone assist to understand how to convert the above to that format in Python? Thanks...RDK
I would say that
0xc0 0xa8 (0x00) 0x01
is 192.168.0.1, your gateway. Seems you've just missed to note that both registers are 16 bits so you've neglected the high byte..
Here is my solution to this problem:
def Decode_TCPIP(reg1,reg2):
# print("Reg1 = "+ reg1 + " Reg2 = " + reg2)
UpperMask = 0xff00
LowerMask = 0x00ff
First = (reg1 & UpperMask)/256
Second = (reg1 & LowerMask)
Third = (reg2 & UpperMask)/256
Forth = (reg2 & LowerMask)
return First, Second, Third, Forth
the returned values are then the four digits in the IP address....RDK
I am trying to write a regex that will parse the following Cisco log messages correctly:
<191>45902: DC-SWITCH2: Aug 30 18:15:16.478: %SFF8472-3-THRESHOLD_VIOLATION: Te0/2: Rx power high warning; Operating value: -0.8 dBm, Threshold value: -1.0 dBm.
Desired output:
Te0/2: Rx power high warning; Operating value: -0.8 dBm, Threshold value: -1.0 dBm.
And:
<191>45902: DC-SWITCH2: Aug 31 19:17:30.147: sensor num : 10 sensor_value :33, high :110 low:85
Desired output:
sensor num : 10 sensor_value :33, high :110 low:85
I have developed the following regex for the first case, but I cannot fathom how to make the mnemonic %STRING section optional:
>\d+:\s.+?:\s.+?(?=:\s):\s%.+?(?=:\s):?\s(.+)
It returns the desired result for the first example, but for the second I get:
10 sensor_value :33, high :110 low:85
You want to make the part that checks for the %STRING non-capturing.
Something like this:
>\d+:\s.+?:\s.+?(?=:\s):\s(?:%.+?:)?\s(.+)
See https://regex101.com/r/F30ALK/1
Why not try something generic like
\d{2}:\d{2}:\d{2}.\d{3}.*? (\b[A-Za-z].*)
where the required output will be in the Group 1.
Example shown here
I am trying to write a program that will read a text file and convert what it reads to another text file but using the given variables. Kinda like a homemade encryption. I want the program to read 2 bytes at a time and read the entire file. I am new to python but enjoy the application. any help would be greatly appreciated
a = 12
b = 34
c = 56
etc... up to 20 different types of variables
file2= open("textfile2.text","w")
file = open("testfile.txt","r")
file.read(2):
if file.read(2) = 12 then;
file2.write("a")
else if file.read(2) = 34
file2.write("b")
else if file.read(2) = 56
file2.write("c")
file.close()
file2.close()
Text file would look like:
1234567890182555
so the program would read 12 and write "a" in the other text file and then read 34 and put "b" in the other text file. Just having some logic issues.
I like your idea here is how I would do it. Note I convert everything to lowercase using lower() however if you understand what I am doing it would be quite simple to extend this to work on both lower and uppercase:
import string
d = dict.fromkeys(string.ascii_lowercase, 0) # Create a dictionary of all the letters in the alphabet
updates = 0
while updates < 20: # Can only encode 20 characters
letter = input("Enter a letter you want to encode or type encode to start encoding the file: ")
if letter.lower() == "encode": # Check if the user inputed encode
break
if len(letter) == 1 and letter.isalpha(): # Check the users input was only 1 character long and in the alphabet
encode = input("What do want to encode %s to: " % letter.lower()) # Ask the user what they want to encode that letter to
d[letter.lower()] = encode
updates += 1
else:
print("Please enter a letter...")
with open("data.txt") as f:
content = list(f.read().lower())
for idx, val in enumerate(content):
if val.isalpha():
content[idx] = d[val]
with open("data.txt", 'w') as f:
f.write(''.join(map(str, content)))
print("The file has been encoded!")
Example Usage:
Original data.txt:
The quick brown fox jumps over the lazy dog
Running the script:
Enter a letter you want to encode or type encode to start encoding the file: T
What do want to encode t to: 6
Enter a letter you want to encode or type encode to start encoding the file: H
What do want to encode h to: 8
Enter a letter you want to encode or type encode to start encoding the file: u
What do want to encode u to: 92
Enter a letter you want to encode or type encode to start encoding the file: 34
Please enter a letter...
Enter a letter you want to encode or type encode to start encoding the file: rt
Please enter a letter...
Enter a letter you want to encode or type encode to start encoding the file: q
What do want to encode q to: 9
Enter a letter you want to encode or type encode to start encoding the file: encode
The file has been encoded!
Encode data.txt:
680 992000 00000 000 092000 0000 680 0000 000
I would read the source file and convert the items as you go into a string. Then write the entire result string separately to the second file. This would also allow you to use the better with open construct for file reading. This allows python to handle file closing for you.
This code will not work because it only reads the first two characters. you need to create your own idea on how to iterate it, but here is an idea (without just making a solution for you)
with open("textfile.text","r") as f:
# you need to create a way to iterate over these two byte/char increments
code = f.read(2)
decoded = <figure out what code translates to>
results += decoded
# now you have a decoded string inside `results`
with open("testfile.txt","w") as f:
f.write(results)
the decoded = <figure out what code translates to> part can be done much better than using a bunch of serial if/elseifs....
perhaps define a dictionary of the encodings?
codings = {
"12": "a",
"45": "b",
# etc...
}
then you could just:
results += codings[code]
instead of the if statements (and it would be faster).
I am writing a simple python script to connect to a SAN via SSH, run a set of commands. Ultimately each command will be logged to a separate log along with a timestamp, and then exit. This is because the device we are connecting to doesn't support certificate ssh connections, and doesn't have decent logging capabilities on its current firmware revision.
The issue that I seem to be running into is that the SSH session that is created seems to be limited to 78 characters wide. The results generated from each command are significantly wider - 155 characters. This is causing a bunch of funkiness.
First, the results in their current state are significantly more difficult to parse. Second, because the buffer is significantly smaller, the final volume command won't execute properly because the pexpect launched SSH session actually gets prompted to "press any key to continue".
How do I change the column width of the pexpect session?
Here is the current code (it works but is incomplete):
#!/usr/bin/python
import pexpect
import os
PASS='mypassword'
HOST='1.2.3.4'
LOGIN_COMMAND='ssh manage#'+HOST
CTL_COMMAND='show controller-statistics'
VDISK_COMMAND='show vdisk-statistics'
VOL_COMMAND='show volume-statistics'
VDISK_LOG='vdisk.log'
VOLUME_LOG='volume.log'
CONTROLLER_LOG='volume.log'
DATE=os.system('date +%Y%m%d%H%M%S')
child=pexpect.spawn(LOGIN_COMMAND)
child.setecho(True)
child.logfile = open('FetchSan.log','w+')
child.expect('Password: ')
child.sendline(PASS)
child.expect('# ')
child.sendline(CTL_COMMAND)
print child.before
child.expect('# ')
child.sendline(VDISK_COMMAND)
print child.before
child.expect('# ')
print "Sending "+VOL_COMMAND
child.sendline(VOL_COMMAND)
print child.before
child.expect('# ')
child.sendline('exit')
child.expect(pexpect.EOF)
print child.before
The output expected:
# show controller-statistics
Durable ID CPU Load Power On Time (Secs) Bytes per second IOPS Number of Reads Number of Writes Data Read Data Written
---------------------------------------------------------------------------------------------------------------------------------------------------------
controller_A 0 45963169 1573.3KB 67 386769785 514179976 6687.8GB 5750.6GB
controller_B 20 45963088 4627.4KB 421 3208370173 587661282 63.9TB 5211.2GB
---------------------------------------------------------------------------------------------------------------------------------------------------------
Success: Command completed successfully.
# show vdisk-statistics
Name Serial Number Bytes per second IOPS Number of Reads Number of Writes Data Read Data Written
------------------------------------------------------------------------------------------------------------------------------------------------
CRS 00c0ff13349e000006d5c44f00000000 0B 0 45861 26756 3233.0MB 106.2MB
DATA 00c0ff1311f300006dd7c44f00000000 2282.4KB 164 23229435 76509765 5506.7GB 1605.3GB
DATA1 00c0ff1311f3000087d8c44f00000000 2286.5KB 167 23490851 78314374 5519.0GB 1603.8GB
DATA2 00c0ff1311f30000c2f8ce5700000000 0B 0 26 4 1446.9KB 65.5KB
FRA 00c0ff13349e000001d8c44f00000000 654.8KB 5 3049980 15317236 1187.3GB 1942.1GB
FRA1 00c0ff13349e000007d9c44f00000000 778.7KB 6 3016569 15234734 1179.3GB 1940.4GB
------------------------------------------------------------------------------------------------------------------------------------------------
Success: Command completed successfully.
# show volume-statistics
Name Serial Number Bytes per second IOPS Number of Reads Number of Writes Data Read Data Written
-----------------------------------------------------------------------------------------------------------------------------------------------------
CRS_v001 00c0ff13349e0000fdd6c44f01000000 14.8KB 5 239611146 107147564 1321.1GB 110.5GB
DATA1_v001 00c0ff1311f30000d0d8c44f01000000 2402.8KB 218 1701488316 336678620 33.9TB 3184.6GB
DATA2_v001 00c0ff1311f3000040f9ce5701000000 0B 0 921 15 2273.7KB 2114.0KB
DATA_v001 00c0ff1311f30000bdd7c44f01000000 2303.4KB 209 1506883611 250984824 30.0TB 2026.6GB
FRA1_v001 00c0ff13349e00001ed9c44f01000000 709.1KB 28 25123082 161710495 1891.0GB 2230.0GB
FRA_v001 00c0ff13349e00001fd8c44f01000000 793.0KB 34 122052720 245322281 3475.7GB 3410.0GB
-----------------------------------------------------------------------------------------------------------------------------------------------------
Success: Command completed successfully.
The output as printed to the terminal (as mentioned, the 3rd command won't execute in its current state):
show controller-statistics
Durable ID CPU Load Power On Time (Secs) Bytes per second
IOPS Number of Reads Number of Writes Data Read
Data Written
----------------------------------------------------------------------
controller_A 3 45962495 3803.1KB
73 386765821 514137947 6687.8GB
5748.9GB
controller_B 20 45962413 5000.7KB
415 3208317860 587434274 63.9TB
5208.8GB
----------------------------------------------------------------------
Success: Command completed successfully.
Sending show volume-statistics
show vdisk-statistics
Name Serial Number Bytes per second IOPS
Number of Reads Number of Writes Data Read Data Written
----------------------------------------------------------------------------
CRS 00c0ff13349e000006d5c44f00000000 0B 0
45861 26756 3233.0MB 106.2MB
DATA 00c0ff1311f300006dd7c44f00000000 2187.2KB 152
23220764 76411017 5506.3GB 1604.1GB
DATA1 00c0ff1311f3000087d8c44f00000000 2295.2KB 154
23481442 78215540 5518.5GB 1602.6GB
DATA2 00c0ff1311f30000c2f8ce5700000000 0B 0
26 4 1446.9KB 65.5KB
FRA 00c0ff13349e000001d8c44f00000000 1829.3KB 14
3049951 15310681 1187.3GB 1941.2GB
FRA1 00c0ff13349e000007d9c44f00000000 1872.8KB 14
3016521 15228157 1179.3GB 1939.5GB
----------------------------------------------------------------------------
Success: Command completed successfully.
Traceback (most recent call last):
File "./fetchSAN.py", line 34, in <module>
child.expect('# ')
File "/Library/Python/2.7/site-packages/pexpect-4.2.1-py2.7.egg/pexpect/spawnbase.py", line 321, in expect
timeout, searchwindowsize, async)
File "/Library/Python/2.7/site-packages/pexpect-4.2.1-py2.7.egg/pexpect/spawnbase.py", line 345, in expect_list
return exp.expect_loop(timeout)
File "/Library/Python/2.7/site-packages/pexpect-4.2.1-py2.7.egg/pexpect/expect.py", line 107, in expect_loop
return self.timeout(e)
File "/Library/Python/2.7/site-packages/pexpect-4.2.1-py2.7.egg/pexpect/expect.py", line 70, in timeout
raise TIMEOUT(msg)
pexpect.exceptions.TIMEOUT: Timeout exceeded.
<pexpect.pty_spawn.spawn object at 0x105333910>
command: /usr/bin/ssh
args: ['/usr/bin/ssh', 'manage#10.254.27.49']
buffer (last 100 chars): '-------------------------------------------------------------\r\nPress any key to continue (Q to quit)'
before (last 100 chars): '-------------------------------------------------------------\r\nPress any key to continue (Q to quit)'
after: <class 'pexpect.exceptions.TIMEOUT'>
match: None
match_index: None
exitstatus: None
flag_eof: False
pid: 19519
child_fd: 5
closed: False
timeout: 30
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: <open file 'FetchSan.log', mode 'w+' at 0x1053321e0>
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
0: re.compile("# ")
And here is what is captured in the log:
Password: mypassword
HP StorageWorks MSA Storage P2000 G3 FC
System Name: Uninitialized Name
System Location:Uninitialized Location
Version:TS230P008
# show controller-statistics
show controller-statistics
Durable ID CPU Load Power On Time (Secs) Bytes per second
IOPS Number of Reads Number of Writes Data Read
Data Written
----------------------------------------------------------------------
controller_A 3 45962495 3803.1KB
73 386765821 514137947 6687.8GB
5748.9GB
controller_B 20 45962413 5000.7KB
415 3208317860 587434274 63.9TB
5208.8GB
----------------------------------------------------------------------
Success: Command completed successfully.
# show vdisk-statistics
show vdisk-statistics
Name Serial Number Bytes per second IOPS
Number of Reads Number of Writes Data Read Data Written
----------------------------------------------------------------------------
CRS 00c0ff13349e000006d5c44f00000000 0B 0
45861 26756 3233.0MB 106.2MB
DATA 00c0ff1311f300006dd7c44f00000000 2187.2KB 152
23220764 76411017 5506.3GB 1604.1GB
DATA1 00c0ff1311f3000087d8c44f00000000 2295.2KB 154
23481442 78215540 5518.5GB 1602.6GB
DATA2 00c0ff1311f30000c2f8ce5700000000 0B 0
26 4 1446.9KB 65.5KB
FRA 00c0ff13349e000001d8c44f00000000 1829.3KB 14
3049951 15310681 1187.3GB 1941.2GB
FRA1 00c0ff13349e000007d9c44f00000000 1872.8KB 14
3016521 15228157 1179.3GB 1939.5GB
----------------------------------------------------------------------------
Success: Command completed successfully.
# show volume-statistics
show volume-statistics
Name Serial Number Bytes per second
IOPS Number of Reads Number of Writes Data Read
Data Written
----------------------------------------------------------------------
CRS_v001 00c0ff13349e0000fdd6c44f01000000 11.7KB
5 239609039 107145979 1321.0GB
110.5GB
DATA1_v001 00c0ff1311f30000d0d8c44f01000000 2604.5KB
209 1701459941 336563041 33.9TB
3183.3GB
DATA2_v001 00c0ff1311f3000040f9ce5701000000 0B
0 921 15 2273.7KB
2114.0KB
DATA_v001 00c0ff1311f30000bdd7c44f01000000 2382.8KB
194 1506859273 250871273 30.0TB
2025.4GB
FRA1_v001 00c0ff13349e00001ed9c44f01000000 1923.5KB
31 25123006 161690520 1891.0GB
2229.1GB
FRA_v001 00c0ff13349e00001fd8c44f01000000 2008.5KB
37 122050872 245301514 3475.7GB
3409.1GB
----------------------------------------------------------------------
Press any key to continue (Q to quit)%
As a starting point: According to the manual, that SAN has a command to disable the pager. See the documentation for set cli-parameters pager off. It may be sufficient to execute that command. It may also have a command to set the terminal rows and columns that it uses for formatting output, although I wasn't able to find one.
Getting to your question: When an ssh client connects to a server and requests an interactive session, it can optionally request a PTY (pseudo-tty) for the server side of the session. When it does that, it informs the server of the lines, columns, and terminal type which the server should use for the TTY. Your SAN may honor PTY requests and use the lines and columns values to format its output. Or it may not.
The ssh client gets the rows and columns for the PTY request from the TTY for its standard input. This is the PTY which pexpect is using to communicate with ssh.
this question discusses how to set the terminal size for a pexpect session. Ssh doesn't honor the LINES or COLUMNS environment variables as far as I can tell, so I doubt that would work. However, calling child.setwinsize() after spawning ssh ought to work:
child = pexpect.spawn(cmd)
child.setwinsize(400,400)
If you have trouble with this, you could try setting the terminal size by invoking stty locally before ssh:
child=pexpect.spawn('stty rows x cols y; ssh user#host')
Finally, you need to make sure that ssh actually requests a PTY for the session. It does this by default in some cases, which should include the way you are running it. But it has a command-line option -tt to force it to allocate a PTY. You could add that option to the ssh command line to make sure:
child=pexpect.spawn('ssh -tt user#host')
or
child=pexpect.spawn('stty rows x cols y; ssh -tt user#host')
my hope for this question (see: bottom) is to lay out as much as I know about the deflate process, and I can receive corrections on areas where I am (perhaps very) misinformed. Hopefully, at the end of it, this question could be a handy resource.
Zlib header
The first two bytes equate to a header for the zlib compression used with the format of (credit)
---CMF--- ---FLG---
0111.1000 1101.0101
CINF -CM- +-||
| |+- FCHECK
| +-- FDICT
+---- FLEVEL
From RFC 1950, right to left:
FCHECK (1.0101) - validates that CMF & FLG as a 16 bit unsigned int is a multiple of 31
FDICT (0) - if set, indicates preset DICT following immediately after FLG
FLEVEL (11) - compression "intensity" [0-3]
CM (1000) - for compression method, where CM = 8 == "deflate" compression method
CINF (0111) - indicates the size of the sliding window used, where CINF = 7 == 32K sliding window
Data block header
The next three bits in the NEW BYTE equate to a header for the Huffman encoded block:
---CMF--- ---FLG--- NEW BYTE
0111.1000 1101.0101 11101100
|-|
| +- BFINAL
+--- BTYPE
From RFC 1951 right to left:
BFINAL (0) - is set (1) if this is the last block of data
BTYPE (10) - Huffman encoding : (00)none; (01)Fixed Huffman codes; (10) dynamic codes; (11) invalid
The Huffman Codes
From here I will work off the assumption of BTYPE = (10)
The following values immediately proceed:
NEW BYTE NXT BYTE
(11101)100 -> 101)(11101) -> 0111111(1
|-|
| +- BFINAL
+--- BTYPE
HLIT (11101) - 5-bit number of length/literal codes, 257 added (257-286)
HDIST (11101) - 5-bit number of distance codes, 1 added (1-32)
HCLEN (1111) - 4-bit number of code-length codes, 4 added (4-19)
Immediately following this is HCLEN(don't forget +4) 3-bit fields, where the values are assigned to this sequence, in order:
16 17 18 0 8 7 9 6 10 5 11 4 12 3 13 2 14 1 15
Since HCLEN = 19, the whole sequence is used
A code length of 0 in that sequence means the corresponding symbol is not used.
As a graphical example, after reading the 19x3 bits we have six extra bits(extra bits in brackets):
NXT BYTE 00000000 00000000 00000000 00000000 00000000 00000000 [000000](00
My Question
Do the last bits in brackets above, get thrown away?
No. The only times in a deflate stream that you skip bits to go to a byte boundary are for a stored block (00), or when the end code in the last block is read. After the bits for the code length code lengths, you continue with the subsequent bits to use the generated Huffman code to read the code lengths.