Consider you have this message (ab,cd,ef) and you have the ROHC (Robust header compression) CRC8 polynomial e0.
C(x) = x^0 + x^1 + x^2 + x^8
Is there any way that I can calculate the CRC on the message backward starting from the last byte and get the same results as if I am calculating it on the original message?
No this is generally not possible for your polynomial (100000111).
EG: 110100111/100000111 = 011010011
but: 111001011/xxxxxxxxx != 011010011 (in general)
However, you can still check for the validity of your message if you know the CRC beforehand.
EG: 110100111/100000111 = 01101001
=> message transmitted = 11010011 01101001
=> message received (reversed) = 10010110 11001011
then: 10010110 11001011/111000001 == 0
(where: 111000001 = reversed(100000111))
=> crc(reversed(11001011)) = crc(11010011) == reversed(10010110) = 01101001
Note that this is only true if the message is reversed BITEWISE.
IE: reversed(ABC) = reversed(101010111100) = 001111010101
= 3D5 = reversed(ABC) != CBA = 110010111010 != reversed(101010111100)
So be careful when implementing your algorithm ;-)
Related
I got the js code below from an archive of hackers delight (view the source)
The code takes in a value (such as 7) and spits out a magic number to multiply with. Then you bitshift to get the results. I don't remember assembly or any math so I'm sure I'm wrong but I can't find the reason why I'm wrong
From my understanding you could get a magic number by writing ceil(1/divide * 1<<32) (or <<64 for 64bit values, but you'd need bigger ints). If you multiple an integer with imul you'd get the result in one register and the remainder in another. The result register is magically the correct result of a division with this magic number from my formula
I wrote some C++ code to show what I mean. However I only tested with the values below. It seems correct. The JS code has a loop and more and I was wondering, why? Am I missing something? What values can I use to get an incorrect result that the JS code would get correctly? I'm not very good at math so I didn't understand any of the comments
#include <cstdio>
#include <cassert>
int main(int argc, char *argv[])
{
auto test_divisor = 7;
auto test_value = 43;
auto a = test_value*test_divisor;
auto b = a-1; //One less test
auto magic = (1ULL<<32)/test_divisor;
if (((1ULL<<32)%test_divisor) != 0) {
magic++; //Round up
}
auto answer1 = (a*magic) >> 32;
auto answer2 = (b*magic) >> 32;
assert(answer1 == test_value);
assert(answer2 == test_value-1);
printf("%lld %lld\n", answer1, answer2);
}
JS code from hackers delight
var two31 = 0x80000000
var two32 = 0x100000000
function magic_signed(d) { with(Math) {
if (d >= two31) d = d - two32// Treat large positive as short for negative.
var ad = abs(d)
var t = two31 + (d >>> 31)
var anc = t - 1 - t%ad // Absolute value of nc.
var p = 31 // Init p.
var q1 = floor(two31/anc) // Init q1 = 2**p/|nc|.
var r1 = two31 - q1*anc // Init r1 = rem(2**p, |nc|).
var q2 = floor(two31/ad) // Init q2 = 2**p/|d|.
var r2 = two31 - q2*ad // Init r2 = rem(2**p, |d|).
do {
p = p + 1;
q1 = 2*q1; // Update q1 = 2**p/|nc|.
r1 = 2*r1; // Update r1 = rem(2**p, |nc|.
if (r1 >= anc) { // (Must be an unsigned
q1 = q1 + 1; // comparison here).
r1 = r1 - anc;}
q2 = 2*q2; // Update q2 = 2**p/|d|.
r2 = 2*r2; // Update r2 = rem(2**p, |d|.
if (r2 >= ad) { // (Must be an unsigned
q2 = q2 + 1; // comparison here).
r2 = r2 - ad;}
var delta = ad - r2;
} while (q1 < delta || (q1 == delta && r1 == 0))
var mag = q2 + 1
if (d < 0) mag = two32 - mag // Magic number and
shift = p - 32 // shift amount to return.
return mag
}}
In the C CODE:
auto magic = (1ULL<<32)/test_divisor;
We get Integer Value in magic because both (1ULL<<32) & test_divisor are Integers.
The Algorithms requires incrementing magic on certain conditions, which is the next conditional statement.
Now, multiplication also gives Integers:
auto answer1 = (a*magic) >> 32;
auto answer2 = (b*magic) >> 32;
C CODE is DONE !
In the JS CODE:
All Variables are var ; no Data types !
No Integer Division ; No Integer Multiplication !
Bitwise Operations are not easy and not suitable to use in this Algorithm.
Numeric Data is via Number & BigInt which are not like "C Int" or "C Unsigned Long Long".
Hence the Algorithm is using loops to Iteratively add and compare whether "Division & Multiplication" has occurred to within the nearest Integer.
Both versions try to Implement the same Algorithm ; Both "should" give same answer, but JS Version is "buggy" & non-standard.
While there are many Issues with the JS version, I will highlight only 3:
(1) In the loop, while trying to get the best Power of 2, we have these two statements :
p = p + 1;
q1 = 2*q1; // Update q1 = 2**p/|nc|.
It is basically incrementing a counter & multiplying a number by 2, which is a left shift in C++.
The C++ version will not require this rigmarole.
(2) The while Condition has 2 Equality comparisons on RHS of || :
while (q1 < delta || (q1 == delta && r1 == 0))
But both these will be false in floating Point Calculations [[ eg check "Math.sqrt(2)*Math.sqrt(0.5) == 1" : even though this must be true, it will almost always be false ]] hence the while Condition is basically the LHS of || , because RHS will always be false.
(3) The JS version returns only one variable mag but user is supposed to get (& use) even variable shift which is given by global variable access. Inconsistent & BAD !
Comparing , we see that the C Version is more Standard, but Point is to not use auto but use int64_t with known number of bits.
First I think ceil(1/divide * 1<<32) can, depending on the divide, have cases where the result is off by one. So you don't need a loop but sometimes you need a corrective factor.
Secondly the JS code seems to allow for other shifts than 32: shift = p - 32 // shift amount to return. But then it never returns that. So not sure what is going on there.
Why not implement the JS code in C++ as well and then run a loop over all int32_t and see if they give the same result? That shouldn't take too long.
And when you find a d where they differ you can then test a / d for all int32_t a using both magic numbers and compare a / d, a * m_ceil and a * m_js.
I want to send a message to the server as per the AIS 140 standard. Please explain how to calculate the checksum. Find below the sample message format.
$Header,iTriangle,KA01I2000,861693034634154,1_37T02B0164MAIS,AIS140,12.976545,N,77.5497 59,E*50
As per AIS140 Standard Checksum is calculated by performing xor to all bytes received from packet.
Note: You have to remove '$'.
Caution: Use data from device to verify this code (Example provided from doc doesnt have valid checksum)
This Javascript code will help your job done.
function checksum(packet) {
const charArray = packet.split('');
let xor = 0;
const n = charArray.length;
for (let i = 1; i < n - 3; i++) {
xor = xor ^ charArray[i].charCodeAt(0);
}
const cs = parseInt("0x" + charArray[n - 2] + charArray[n - 1]);
return xor === cs;
}
checksum('$Header,iTriangle,KA01I2000,861693034634154,1_37T02B0164MAIS,AIS140,12.976545,N,77.549759,E*50')
I found this snippet on 'codefights' submitted by a programmer. My solution to the problem was 30 lines, whereas this is just a beauty.
But I am not able to understand the logic.
Can anyone explain this.
int mirrorBits(int a) {
int r = 0;
for (; a; a >>= 1)
r = r << 1 | a & 1;
return r;
}
input a = 8; output : 1
First of all, there is a very good StackOverflow answer here:
Most Efficient Algorithm for Bit Reversal ( from MSB->LSB to LSB->MSB) in C
The algorithm makes use of
>> ... binary shift right (100b >> 1 == 10b)
<< ... binary shift left (100b << 1 == 1000b
| .... binary or (100b | 10b == 110b)
& .... binary and (111b & 100b == 100b)
The for loop shifts a to the right until all bits have fallen out of a.
Imagine you start with a = 101101 then a >>= 1 does the following:
At the end of loop 1: a == 10110
At the end of loop 2: a == 01011
At the end of loop 3: a == 00101
At the end of loop 4: a == 00010
At the end of loop 5: a == 00001
At the end of loop 6: a == 00000 (condition fails -> loop ends)
The body of the loop shifts b one bit right, uses & to mask the last bit of a and adds it as last digit to b. The or can be used to add the last digit because << inserts 0 for all "new" bits.
Imagine you start with a = 101101
loop 1: a = 101101, r = 0 => 01
loop 2: a = 010110, r = 01 => 010
loop 3: a = 001011, r = 010 => 0101
loop 4: a = 000101, r = 0101 => 01011
loop 5: a = 000010, r = 01011 => 010110
loop 6: a = 000001, r = 010110 => 0101101
In detail the inner loop #3 does the following:
(a is 001011 and r is 010)
r << 1 changes r from 010 to 0100. The last digit is the inserted 0.
a & 1 masks the current last bit from a (the 1 in 001011)
now we have (0100 | 1) which has the result 0101.
Warning: This algorithm is not really mirroring the bits, because you do not get the original value if you apply the algorithm to the result.
If you need a mirrored 32-bit unsigned integer you have to loop 32 times independently of the value of a:
unsigned int r = 0;
unsigned int a = 12345;
for(int i = 0; i < 32; ++i)
{
r = (r << 1) | (a & 1);
a >>= 1;
}
If you apply this algorithm twice, you should get the original value.
I am making a program for nth Fibonacci number. I made the following program using recursion and memoization.
The main problem is that the value of n can go up to 10000 which means that the Fibonacci number of 10000 would be more than 2000 digit long.
With a little bit of googling, I found that i could use arrays and store every digit of the solution in an element of the array but I am still not able to figure out how to implement this approach with my program.
#include<iostream>
using namespace std;
long long int memo[101000];
long long int n;
long long int fib(long long int n)
{
if(n==1 || n==2)
return 1;
if(memo[n]!=0)
return memo[n];
return memo[n] = fib(n-1) + fib(n-2);
}
int main()
{
cin>>n;
long long int ans = fib(n);
cout<<ans;
}
How do I implement that approach or if there is another method that can be used to achieve such large values?
One thing that I think should be pointed out is there's other ways to implement fib that are much easier for something like C++ to compute
consider the following pseudo code
function fib (n) {
let a = 0, b = 1, _;
while (n > 0) {
_ = a;
a = b;
b = b + _;
n = n - 1;
}
return a;
}
This doesn't require memoisation and you don't have to be concerned about blowing up your stack with too many recursive calls. Recursion is a really powerful looping construct but it's one of those fubu things that's best left to langs like Lisp, Scheme, Kotlin, Lua (and a few others) that support it so elegantly.
That's not to say tail call elimination is impossible in C++, but unless you're doing something to optimise/compile for it explicitly, I'm doubtful that whatever compiler you're using would support it by default.
As for computing the exceptionally large numbers, you'll have to either get creative doing adding The Hard Way or rely upon an arbitrary precision arithmetic library like GMP. I'm sure there's other libs for this too.
Adding The Hard Way™
Remember how you used to add big numbers when you were a little tater tot, fresh off the aluminum foil?
5-year-old math
1259601512351095520986368
+ 50695640938240596831104
---------------------------
?
Well you gotta add each column, right to left. And when a column overflows into the double digits, remember to carry that 1 over to the next column.
... <-001
1259601512351095520986368
+ 50695640938240596831104
---------------------------
... <-472
The 10,000th fibonacci number is thousands of digits long, so there's no way that's going to fit in any integer C++ provides out of the box. So without relying upon a library, you could use a string or an array of single-digit numbers. To output the final number, you'll have to convert it to a string tho.
(woflram alpha: fibonacci 10000)
Doing it this way, you'll perform a couple million single-digit additions; it might take a while, but it should be a breeze for any modern computer to handle. Time to get to work !
Here's an example in of a Bignum module in JavaScript
const Bignum =
{ fromInt: (n = 0) =>
n < 10
? [ n ]
: [ n % 10, ...Bignum.fromInt (n / 10 >> 0) ]
, fromString: (s = "0") =>
Array.from (s, Number) .reverse ()
, toString: (b) =>
b .reverse () .join ("")
, add: (b1, b2) =>
{
const len = Math.max (b1.length, b2.length)
let answer = []
let carry = 0
for (let i = 0; i < len; i = i + 1) {
const x = b1[i] || 0
const y = b2[i] || 0
const sum = x + y + carry
answer.push (sum % 10)
carry = sum / 10 >> 0
}
if (carry > 0) answer.push (carry)
return answer
}
}
We can verify that the Wolfram Alpha answer above is correct
const { fromInt, toString, add } =
Bignum
const bigfib = (n = 0) =>
{
let a = fromInt (0)
let b = fromInt (1)
let _
while (n > 0) {
_ = a
a = b
b = add (b, _)
n = n - 1
}
return toString (a)
}
bigfib (10000)
// "336447 ... 366875"
Expand the program below to run it in your browser
const Bignum =
{ fromInt: (n = 0) =>
n < 10
? [ n ]
: [ n % 10, ...Bignum.fromInt (n / 10 >> 0) ]
, fromString: (s = "0") =>
Array.from (s) .reverse ()
, toString: (b) =>
b .reverse () .join ("")
, add: (b1, b2) =>
{
const len = Math.max (b1.length, b2.length)
let answer = []
let carry = 0
for (let i = 0; i < len; i = i + 1) {
const x = b1[i] || 0
const y = b2[i] || 0
const sum = x + y + carry
answer.push (sum % 10)
carry = sum / 10 >> 0
}
if (carry > 0) answer.push (carry)
return answer
}
}
const { fromInt, toString, add } =
Bignum
const bigfib = (n = 0) =>
{
let a = fromInt (0)
let b = fromInt (1)
let _
while (n > 0) {
_ = a
a = b
b = add (b, _)
n = n - 1
}
return toString (a)
}
console.log (bigfib (10000))
Try not to use recursion for a simple problem like fibonacci. And if you'll only use it once, don't use an array to store all results. An array of 2 elements containing the 2 previous fibonacci numbers will be enough. In each step, you then only have to sum up those 2 numbers. How can you save 2 consecutive fibonacci numbers? Well, you know that when you have 2 consecutive integers one is even and one is odd. So you can use that property to know where to get/place a fibonacci number: for fib(i), if i is even (i%2 is 0) place it in the first element of the array (index 0), else (i%2 is then 1) place it in the second element(index 1). Why can you just place it there? Well when you're calculating fib(i), the value that is on the place fib(i) should go is fib(i-2) (because (i-2)%2 is the same as i%2). But you won't need fib(i-2) any more: fib(i+1) only needs fib(i-1)(that's still in the array) and fib(i)(that just got inserted in the array).
So you could replace the recursion calls with a for loop like this:
int fibonacci(int n){
if( n <= 0){
return 0;
}
int previous[] = {0, 1}; // start with fib(0) and fib(1)
for(int i = 2; i <= n; ++i){
// modulo can be implemented with bit operations(much faster): i % 2 = i & 1
previous[i&1] += previous[(i-1)&1]; //shorter way to say: previous[i&1] = previous[i&1] + previous[(i-1)&1]
}
//Result is in previous[n&1]
return previous[n&1];
}
Recursion is actually discommanded while programming because of the time(function calls) and ressources(stack) it consumes. So each time you use recursion, try to replace it with a loop and a stack with simple pop/push operations if needed to save the "current position" (in c++ one can use a vector). In the case of the fibonacci, the stack isn't even needed but if you are iterating over a tree datastructure for example you'll need a stack (depends on the implementation though). As I was looking for my solution, I saw #naomik provided a solution with the while loop. That one is fine too, but I prefer the array with the modulo operation (a bit shorter).
Now concerning the problem of the size long long int has, it can be solved by using external libraries that implement operations for big numbers (like the GMP library or Boost.multiprecision). But you could also create your own version of a BigInteger-like class from Java and implement the basic operations like the one I have. I've only implemented the addition in my example (try to implement the others they are quite similar).
The main idea is simple, a BigInt represents a big decimal number by cutting its little endian representation into pieces (I'll explain why little endian at the end). The length of those pieces depends on the base you choose. If you want to work with decimal representations, it will only work if your base is a power of 10: if you choose 10 as base each piece will represent one digit, if you choose 100 (= 10^2) as base each piece will represent two consecutive digits starting from the end(see little endian), if you choose 1000 as base (10^3) each piece will represent three consecutive digits, ... and so on. Let's say that you have base 100, 12765 will then be [65, 27, 1], 1789 will be [89, 17], 505 will be [5, 5] (= [05,5]), ... with base 1000: 12765 would be [765, 12], 1789 would be [789, 1], 505 would be [505]. It's not the most efficient, but it is the most intuitive (I think ...)
The addition is then a bit like the addition on paper we learned at school:
begin with the lowest piece of the BigInt
add it with the corresponding piece of the other one
the lowest piece of that sum(= the sum modulus the base) becomes the corresponding piece of the final result
the "bigger" pieces of that sum will be added ("carried") to the sum of the following pieces
go to step 2 with next piece
if no piece left, add the carry and the remaining bigger pieces of the other BigInt (if it has pieces left)
For example:
9542 + 1097855 = [42, 95] + [55, 78, 09, 1]
lowest piece = 42 and 55 --> 42 + 55 = 97 = [97]
---> lowest piece of result = 97 (no carry, carry = 0)
2nd piece = 95 and 78 --> (95+78) + 0 = 173 = [73, 1]
---> 2nd piece of final result = 73
---> remaining: [1] = 1 = carry (will be added to sum of following pieces)
no piece left in first `BigInt`!
--> add carry ( [1] ) and remaining pieces from second `BigInt`( [9, 1] ) to final result
--> first additional piece: 9 + 1 = 10 = [10] (no carry)
--> second additional piece: 1 + 0 = 1 = [1] (no carry)
==> 9542 + 1 097 855 = [42, 95] + [55, 78, 09, 1] = [97, 73, 10, 1] = 1 107 397
Here is a demo where I used the class above to calculate the fibonacci of 10000 (result is too big to copy here)
Good luck!
PS: Why little endian? For the ease of the implementation: it allows to use push_back when adding digits and iteration while implementing the operations will start from the first piece instead of the last piece in the array.
In practice, I select an executable. Size 20Mb.
I read the content using file.read(size=16).
If length of the returned byte string is less than 16 I fill the rest with \0 (NULL).
f = open("./installer.exe","rb")
obj = AES.new(b"0123456789012345",AES.MODE_CBC, b"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0")
bs = b""
t = f.read(16)
while t != b"":
if len(t) < 16:
t = t + b"\0" * (16 - len(t)) # if < 16 bytes using padding
bs = bs + obj.encrypt(t)
else:
bs = bs + obj.encrypt(t)
t = f.read(16)
then, bs contents the byte string of ALL content encrypted with 0123456789012345
I realise the mechanism of reading file first, then I encrypt the content as seen in the above piece of code (using obj.encrypt()). Then I write a new file with the content encrypted. The I read the data of encrypted file and by a similar procedure decrypt the data using obj.decrypt in intervals of 16 bytes and then I write a new file with the decrypted data.
This takes approximately 3 minutes.
¿It's fast, slow, or expected?
According to what I saw, the module is written in C. ¿Maybe should I use Cython embedded to make it faster?
How PGP can supposedly decrypt higher amounts of data in real time, for example, in an encrypted virtual disk?
edit:
This take almost same:
obj = AES.new(b"0123456789012345",AES.MODE_CBC, b"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0")
bs = b""
t = f.read(16)
while t != b"":
if len(t) < 16:
t = t + b"\0" * (16 - len(t))
bs = bs + t
else:
bs = bs + t
t = f.read(16)
bse = obj.encrypt(bs)
Ok. The problem was the size of the buffers encrypted. I decided to use 64000 bytes strings.
The procedure its simple. total size / string segments -> encrypt. And in the last segment, if the size of the segment is inferior to 64000 AND NOT multiple of 16, finds the nearest multiple and the remaining space is filled
bs = b""
dt = f.read()
dtl = len(dt)
dtr = ( dtl / 64000 ) + 1
for x in range(0, dtr):
if x == dtr-1:
i1 = 64000 * x
dst = dtl - i1
i = math.ceil(dst / 16.0) * 16
dst = i - dst
buf = dt[i1:] + (b"\0" * int(dst))
bs = bs + obj.encrypt(buf)
else:
i1, i2 = 64000 * x , 64000 * (x+1)
bs = bs + obj.encrypt(dt[i1:i2])
Now takes 10 seconds.
Thanks for all.