While working with the following PDF, there is an example in
Section 4: CRC-16 Code and Example
(page 95 or 91) that shows a serial packet with a CRC16 value of 133 (LSB) and 24 (MSB).
However, I have tried different calculators, for example:
Lammert
Elaborate calculator
CRC calc
but I cannot get the CRC16 values that the PDF indicates, regardless of the byte combination I use.
How can I correctly calculate the CRC16 in the example, preferably using one of these calculators? (otherwise, C/C++ code should work).
Thanks.
This particular CRC is CRC-16/ARC. crcany generates the code for this CRC, which includes this simple bit-wise routine:
#include <stddef.h>
#include <stdint.h>
uint16_t crc16arc_bit(uint16_t crc, void const *mem, size_t len) {
unsigned char const *data = mem;
if (data == NULL)
return 0;
for (size_t i = 0; i < len; i++) {
crc ^= data[i];
for (unsigned k = 0; k < 8; k++) {
crc = crc & 1 ? (crc >> 1) ^ 0xa001 : crc >> 1;
}
}
return crc;
}
The standard interface is to do crc = crc16arc_bit(0, NULL, 0); to get the initial value (zero in this case), and then crc = crc16arc_bit(crc, data, len); with successive portions of the message to compute the CRC.
If you do that on the nine-byte message in the appendix, {1, 2, 1, 0, 17, 3, 'M', 'O', 'C'}, the returned CRC is 0x1885, which has the least significant byte 133 in decimal and most significant byte 24 in decimal.
Faster table-driven routines are also generated by crcany.
If you give 01 02 01 00 11 03 4d 4f 43 as hex to Lammert Bies' calculator, the very first one, called "CRC-16" gives 0x1885.
If you give 0102010011034d4f43 to crccalc.com and hit Calc-CRC16, the second line is CRC-16/ARC, with the result 0x1885.
Related
I'm building something which uses the DS18B20 temperature sensor.
First I am trying to understand the example CRC in the Maxim application note 27, "Understanding and Using Cyclic Redundancy Checks with Maxim iButton Products" (https://www.analog.com/en/technical-articles/understanding-and-using-cyclic-redundancy-checks-with-maxim-1wire-and-ibutton-products.html).
It doesn't look too hard to code the conversion but my problem is that I cannot find any calculator that gives me the correct answer of 0xA2.
On page 5, example 2 the complete ROM code is given in hex as A2=(CRC), 00 00 00 01 B8 1C 02=(Family code).
The generator polynomial is 100110001 (X8+X5+X4+1).
On the site https://crccalc.com/ it has a CRC-8/MAXIM algorithm which has the correct generator but the RefIn and RefOut are both true whereas I cannot see anything in the application note about reversing parts (although I have tried this).
On the site https://tomeko.net/online_tools/crc8.php?lang=en it claims to implement the CRC from the application note but it gives the same answers as crccalc.com. Also note that crccalc has the same lookup table for the Maxim algorithm as the application note so no surprise the two web sites are giving the same answers.
Finally I found a site, https://www.rndtool.info/CRC-step-by-step-calculator/, that allows me to add the polynomial and bit stream in binary and it shows the 'hand' calculation of the CRC. This says nothing about input and output refs so I assume they are false. This gives different answers to the other two sites probably because of the ref values but still does not give 0xA2.
Has anyone correctly calculated the given value in the application note?
I don't want to start programming until I understand what is going on and I cannot read data from a device if I cannot decipher the CRC correctly. This is driving me mad at the moment. I've tried the number reflected and forwards with the generator reflected and forwards plus reversing the answer but I never get 0xA2.
You didn't give a language in your tags. Here is an example in C:
#include <stdio.h>
unsigned crc8maximdow(unsigned char *data, size_t len) {
unsigned crc = 0;
for (size_t i = 0; i < len; i++) {
crc ^= data[i];
for (unsigned k = 0; k < 8; k++)
crc = crc & 1 ? (crc >> 1) ^ 0x8c : crc >> 1;
}
return crc;
}
int main(void) {
unsigned char data[] = {2, 0x1c, 0xb8, 1, 0, 0, 0};
printf("0x%02x\n", crc8maximdow(data, sizeof(data)));
return 0;
}
That prints 0xa2.
Using Mark's answer above I worked out what I was doing wrong with the Maxim example. Tutorials were talking about 'reflecting' the input data so I literally took the 56-bit bit pattern and reversed it which gave the wrong answer in the site https://crccalc.com/. Instead I reversed the hex numbers so that 00 00 00 01 B8 1C 02 becomes 02 1C B8 01 00 00 00 which gives the correct CRC of 0xA2 on the website. Hope this may help anyone else having the same problem.
I'm using a 24 bit I2C ADC with the Arduino and there is no 3 byte (24 bit) data type so I instead used the uint32_t which is a 32 bit unsigned int. My actual output however, is a 24 bit signed number as you can see below:
Also here is the code that I used to read the results if you're interested:
uint32_t readData(){
Wire.beginTransmission(address);
Wire.write(0x10);
Wire.endTransmission();
Wire.requestFrom(address,3);
byte dataMSB = Wire.read();
byte data = Wire.read();
byte dataLSB = Wire.read();
uint32_t data32 = dataMSB;
data32 <<= 8;
data32 |= data;
data32 <<= 8;
data32 |= dataLSB;
return data32;
}
In order for this number to be useful, I need to convert it back to a 24 bit signed integer (I'm not sure how to do that or eve if it's possible because 24 is not a power of 2) so I'm a bit stuck. It would be great if somebody can help me as I'm almost finished with the project and this is one of the last few steps.
The problem is that there’s no safe and portable way to use shifting for sign extension in C — at best it is implementation defined. So if you want to do it portably, you need to convert your 2s-complement value manually into a signed integer.
int32_t cvt24bit(uint32_t val) {
val &= 0xffffff; // limit to 24 bits -- may not be necessary
if (val >= (UINT32_C(1) << 23))
return (int32_t)val - (INT32_C(1) << 24);
else
return val;
}
this will take your 24-bit two’s-complement value in a uint32_t and convert it to a (signed) int32_t.
Conversion from 24-bit two’s complement in a uint32_t to int32_t can be done with:
int32_t Convert(uint32_t x)
{
int32_t t = x & 0xffffff;
return t - (t >> 23 << 24);
}
The x & 0xffffff ensures the number has no spurious bits above bit 23. If it is certain no such bits are set, then the statement can be just int32_t t = x;.
Then t >> 23 removes bits 0 to 22, leave just bit 23, which is the sign bit for a 24-bit integer. Then << 24 scales this, producing either 0 (for positive numbers) or 224 (for negative numbers). Subtracting that from t produces the desired value.
Use int32_t instead of unit32_t for data32. Then before returning the value, shift it left by 8, then right by 8 to sign extend it.
So this code:
uint32_t readData(){
Wire.beginTransmission(address);
Wire.write(0x10);
Wire.endTransmission();
Wire.requestFrom(address,3);
byte dataMSB = Wire.read();
byte data = Wire.read();
byte dataLSB = Wire.read();
int32_t data32 = dataMSB;
data32 <<= 8;
data32 |= data;
data32 <<= 8;
data32 |= dataLSB;
return (data32 << 8) >> 8;
}
I run a bench of computations using SIMD intructions. These instructions return a vector of 16 bytes as result, named compare, with each byte being 0x00 or 0xff :
0 1 2 3 4 5 6 7 15 16
compare : 0x00 0x00 0x00 0x00 0xff 0x00 0x00 0x00 ... 0xff 0x00
Bytes set to 0xff mean I need to run the function do_operation(i) with i being the position of the byte.
For instance, the above compare vector mean, I need to run this sequence of operations :
do_operation(4);
do_operation(15);
Here is the fastest solution I came up with until now :
for(...) {
//
// SIMD computations
//
__m128i compare = ... // Result of SIMD computations
// Extract high and low quadwords for compare vector
std::uint64_t cmp_low = (_mm_cvtsi128_si64(compare));
std::uint64_t cmp_high = (_mm_extract_epi64(compare, 1));
// Process low quadword
if (cmp_low) {
const std::uint64_t low_possible_positions = 0x0706050403020100;
const std::uint64_t match_positions = _pext_u64(
low_possible_positions, cmp_low);
const int match_count = _popcnt64(cmp_low) / 8;
const std::uint8_t* match_pos_array =
reinterpret_cast<const std::uint8_t*>(&match_positions);
for (int i = 0; i < match_count; ++i) {
do_operation(i);
}
}
// Process high quadword (similarly)
if (cmp_high) {
const std::uint64_t high_possible_positions = 0x0f0e0d0c0b0a0908;
const std::uint64_t match_positions = _pext_u64(
high_possible_positions, cmp_high);
const int match_count = _popcnt64(cmp_high) / 8;
const std::uint8_t* match_pos_array =
reinterpret_cast<const std::uint8_t*>(&match_positions);
for(int i = 0; i < match_count; ++i) {
do_operation(i);
}
}
}
I start with extracting the first and second 64 bits integers of the 128 bits vector (cmp_low and cmp_high). Then I use popcount to compute the number of bytes set to 0xff (number of bits set to 1 divided by 8). Finally, I use pext to get positions, without zeros, like this :
0x0706050403020100
0x000000ff00ff0000
|
PEXT
|
0x0000000000000402
I would like to find a faster solution to extract the positions of the bytes set to 0xff in the compare vector. More precisely, the are very often only 0, 1 or 2 bytes set to 0xff in the compare vector and I would like to use this information to avoid some branches.
Here's a quick outline of how you could reduce the number of tests:
First use a function to project all the lsb or msb of each byte of your 128bit integer into a 16bit value (for instance, there's a SSE2 assembly instruction for that on X86 cpus: pmovmskb, which is supported on Intel and MS compilers with the _mm_movemask_pi8 intrinsic, and gcc has also an intrinsic: __builtin_ia32_ppmovmskb128, );
Then split that value in 4 nibbles;
define functions to handle each possible values of a nibble (from 0 to 15) and put these in an array;
Finally call the function indexed by each nibble (with extra parameters to indicate which nibble in the 16bits it is).
Since in your case very often only 0, 1 or 2 bytes are set to 0xff in the compare vector, a short
while-loop on the bitmask might be more efficient than a solution based on the pext
instruction. See also my answer on a similar question.
/*
gcc -O3 -Wall -m64 -mavx2 -march=broadwell esbsimd.c
*/
#include <stdio.h>
#include <immintrin.h>
int do_operation(int i){ /* some arbitrary do_operation() */
printf("i = %d\n",i);
return 0;
}
int main(){
__m128i compare = _mm_set_epi8(0xFF,0,0,0, 0,0,0,0, 0,0,0,0xFF, 0,0,0,0); /* Take some randon value for compare */
int k = _mm_movemask_epi8(compare);
while (k){
int i=_tzcnt_u32(k); /* Count the number of trailing zero bits in k. BMI1 instruction set, Haswell or newer. */
do_operation(i);
k=_blsr_u32(k); /* Clear the lowest set bit in k. */
}
return 0;
}
/*
Output:
i = 4
i = 15
*/
If I have a char array A, I use it to store hex
A = "0A F5 6D 02" size=11
The binary representation of this char array is:
00001010 11110101 01101101 00000010
I want to ask is there any function can random flip the bit?
That is:
if the parameter is 5
00001010 11110101 01101101 00000010
-->
10001110 11110001 01101001 00100010
it will random choose 5 bit to flip.
I am trying make this hex data to binary data and use bitmask method to achieve my requirement. Then turn it back to hex. I am curious is there any method to do this job more quickly?
Sorry, my question description is not clear enough. In simply, I have some hex data, and I want to simulate bit error in these data. For example, if I have 5 byte hex data:
"FF00FF00FF"
binary representation is
"1111111100000000111111110000000011111111"
If the bit error rate is 10%. Then I want to make these 40 bits have 4 bits error. One extreme random result: error happened in the first 4 bit:
"0000111100000000111111110000000011111111"
First of all, find out which char the bit represents:
param is your bit to flip...
char *byteToWrite = &A[sizeof(A) - (param / 8) - 1];
So that will give you a pointer to the char at that array offset (-1 for 0 array offset vs size)
Then get modulus (or more bit shifting if you're feeling adventurous) to find out which bit in here to flip:
*byteToWrite ^= (1u << param % 8);
So that should result for a param of 5 for the byte at A[10] to have its 5th bit toggled.
store the values of 2^n in an array
generate a random number seed
loop through x times (in this case 5) and go data ^= stored_values[random_num]
Alternatively to storing the 2^n values in an array, you could do some bit shifting to a random power of 2 like:
data ^= (1<<random%7)
Reflecting the first comment, you really could just write out that line 5 times in your function and avoid the overhead of a for loop entirely.
You have 32 bit number. You can treate the bits as parts of hte number and just xor this number with some random 5-bits-on number.
int count_1s(int )
{
int m = 0x55555555;
int r = (foo&m) + ((foo>>>1)&m);
m = 0x33333333;
r = (r&m) + ((r>>>2)&m);
m = 0x0F0F0F0F;
r = (r&m) + ((r>>>4)&m);
m = 0x00FF00FF;
r = (r&m) + ((r>>>8)&m);
m = 0x0000FFFF;
return r = (r&m) + ((r>>>16)&m);
}
void main()
{
char input[] = "0A F5 6D 02";
char data[4] = {};
scanf("%2x %2x %2x %2x", &data[0], &data[1], &data[2], &data[3]);
int *x = reinterpret_cast<int*>(data);
int y = rand();
while(count_1s(y) != 5)
{
y = rand(); // let's have this more random
}
*x ^= y;
printf("%2x %2x %2x %2x" data[0], data[1], data[2], data[3]);
return 0;
}
I see no reason to convert the entire string back and forth from and to hex notation. Just pick a random character out of the hex string, convert this to a digit, change it a bit, convert back to hex character.
In plain C:
#include <stdio.h>
#include <stdlib.h>
#include <sys/time.h>
int main (void)
{
char *hexToDec_lookup = "0123456789ABCDEF";
char hexstr[] = "0A F5 6D 02";
/* 0. make sure we're fairly random */
srand(time(0));
/* 1. loop 5 times .. */
int i;
for (i=0; i<5; i++)
{
/* 2. pick a random hex digit
we know it's one out of 8, grouped per 2 */
int hexdigit = rand() & 7;
hexdigit += (hexdigit>>1);
/* 3. convert the digit to binary */
int hexvalue = hexstr[hexdigit] > '9' ? hexstr[hexdigit] - 'A'+10 : hexstr[hexdigit]-'0';
/* 4. flip a random bit */
hexvalue ^= 1 << (rand() & 3);
/* 5. write it back into position */
hexstr[hexdigit] = hexToDec_lookup[hexvalue];
printf ("[%s]\n", hexstr);
}
return 0;
}
It might even be possible to omit the convert-to-and-from-ASCII steps -- flip a bit in the character string, check if it's still a valid hex digit and if necessary, adjust.
First randomly chose x positions (each position consist of array index and the bit position).
Now if you want to flip ith bit from right for a number n. Find the remainder of n by 2n as :
code:
int divisor = (2,i);
int remainder = n % divisor;
int quotient = n / divisor;
remainder = (remainder == 0) ? 1 : 0; // flip the remainder or the i th bit from right.
n = divisor * quotient + remainder;
Take mod 8 of input(5%8)
Shift 0x80 to right by input value (e.g 5)
XOR this value with (input/8)th element of your character array.
code:
void flip_bit(int bit)
{
Array[bit/8] ^= (0x80>>(bit%8));
}
I have run into an interesting problem lately:
Lets say I have an array of bytes (uint8_t to be exact) of length at least one. Now i need a function that will get a subsequence of bits from this array, starting with bit X (zero based index, inclusive) and having length L and will return this as an uint32_t. If L is smaller than 32 the remaining high bits should be zero.
Although this is not very hard to solve, my current thoughts on how to do this seem a bit cumbersome to me. I'm thinking of a table of all the possible masks for a given byte (start with bit 0-7, take 1-8 bits) and then construct the number one byte at a time using this table.
Can somebody come up with a nicer solution? Note that i cannot use Boost or STL for this - and no, it is not a homework, its a problem i run into at work and we do not use Boost or STL in the code where this thing goes. You can assume that: 0 < L <= 32 and that the byte array is large enough to hold the subsequence.
One example of correct input/output:
array: 00110011 1010 1010 11110011 01 101100
subsequence: X = 12 (zero based index), L = 14
resulting uint32_t = 00000000 00000000 00 101011 11001101
Only the first and last bytes in the subsequence will involve some bit slicing to get the required bits out, while the intermediate bytes can be shifted in whole into the result. Here's some sample code, absolutely untested -- it does what I described, but some of the bit indices could be off by one:
uint8_t bytes[];
int X, L;
uint32_t result;
int startByte = X / 8, /* starting byte number */
startBit = 7 - X % 8, /* bit index within starting byte, from LSB */
endByte = (X + L) / 8, /* ending byte number */
endBit = 7 - (X + L) % 8; /* bit index within ending byte, from LSB */
/* Special case where start and end are within same byte:
just get bits from startBit to endBit */
if (startByte == endByte) {
uint8_t byte = bytes[startByte];
result = (byte >> endBit) & ((1 << (startBit - endBit)) - 1);
}
/* All other cases: get ending bits of starting byte,
all other bytes in between,
starting bits of ending byte */
else {
uint8_t byte = bytes[startByte];
result = byte & ((1 << startBit) - 1);
for (int i = startByte + 1; i < endByte; i++)
result = (result << 8) | bytes[i];
byte = bytes[endByte];
result = (result << (8 - endBit)) | (byte >> endBit);
}
Take a look at std::bitset and boost::dynamic_bitset.
I would be thinking something like loading a uint64_t with a cast and then shifting left and right to lose the uninteresting bits.
uint32_t extract_bits(uint8_t* bytes, int start, int count)
{
int shiftleft = 32+start;
int shiftright = 64-count;
uint64_t *ptr = (uint64_t*)(bytes);
uint64_t hold = *ptr;
hold <<= shiftleft;
hold >>= shiftright;
return (uint32_t)hold;
}
For the sake of completness, i'am adding my solution inspired by the comments and answers here. Thanks to all who bothered to think about the problem.
static const uint8_t firstByteMasks[8] = { 0xFF, 0x7F, 0x3F, 0x1F, 0x0F, 0x07, 0x03, 0x01 };
uint32_t getBits( const uint8_t *buf, const uint32_t bitoff, const uint32_t len, const uint32_t bitcount )
{
uint64_t result = 0;
int32_t startByte = bitoff / 8; // starting byte number
int32_t endByte = ((bitoff + bitcount) - 1) / 8; // ending byte number
int32_t rightShift = 16 - ((bitoff + bitcount) % 8 );
if ( endByte >= len ) return -1;
if ( rightShift == 16 ) rightShift = 8;
result = buf[startByte] & firstByteMasks[bitoff % 8];
result = result << 8;
for ( int32_t i = startByte + 1; i <= endByte; i++ )
{
result |= buf[i];
result = result << 8;
}
result = result >> rightShift;
return (uint32_t)result;
}
Few notes: i tested the code and it seems to work just fine, however, there may be bugs. If i find any, i will update the code here. Also, there are probably better solutions!