Encoding numbers in BCD (Casio serial interface) [closed] - c++

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I am attempting to create a device that talks to a Casio fx-9750 calculator through its serial port with an Arduino. I have figured out how to receive values and decode the BCD, but I'm stuck on how to create the required values from a float (to transmit back).
The calculator sends a data packet, which has an exponent value, several data values, and a byte that contains information about negativity, imaginary parts, etc. Each data value is worth one hundredth of the previous one, so the first is the amount of 10s, the next the amount of 0.1s, the next the amount of 0.001s, etc. This continues on until the 0.0000000000001s, though this is out of the range of what I'll really need, so that level of accuracy is not really important to me. The output of my receiving program looks like this:
Exponent: 1
10s: 1
0.1s: 23
0.001s: 40
This represents 12.34.
The general equation I worked out was: (let a=10s, b=0.1s, e=exponent etc)
((a*10)+(b*0.1)+(c*0.001))*10^(E-1)
If the exponent were to change to two:
Exponent: 2
10s: 1
0.1s: 23
0.001s: 40
This would represent 123.4
This method of dropping by hundredths each time is presumably used because they can store two digits in each byte with BCD, so it is most efficient to let each row have two digits as each row is stored as one byte.
I have come up with an equation that can calculate the exponent by counting the amount of digits before the decimal point and subtracting two, however this seems messy as it involves strings. I think a purely mathematical solution would be more elegant, if it is possible.
What is the fastest and simplest way to go from a normal number (e.g. 123.4) into this arrangement?
A solution in Arduino language would be greatly appreciated, but any insight whatsoever into the mathematical process needed would be equally valued.
Edit regarding floats:
I should clarify - I will be dealing with floats in other parts of my program and would like my inputted values to be compatible with numbers of any size (within reason, as stated before). I have no problem with multiplying them to be ints or casting them as other datatypes.

Hah, that was fun!
#include <stdio.h>
#include <assert.h>
#include <math.h>
#include <float.h>
struct num_s {
// exponent
int e;
// v[0] is *10
// v[1] is *0.01
// v[2] is *0.0001
// and so on...
// to increase precision increase array count
int v[6];
};
#define NUM_VALSCNT (sizeof(((struct num_s*)0)->v)/sizeof(((struct num_s*)0)->v[0]))
// creates num_s object from a double
struct num_s num_create(double v) {
struct num_s t;
// find exponent so that v <= 10
t.e = 0;
while (fabs(v) >= 10.0) {
++t.e;
v /= 10.0;
}
// for each output number get the integral part of number
// then multiply the rest by 100 and continue
for (size_t i = 0; i < sizeof(t.v) / sizeof(t.v[0]); ++i) {
const double tmp = fmod(v, 1.0);
t.v[i] = v - tmp;
v = tmp * 100;
}
return t;
}
// converts back from num object to double
double num_get(struct num_s t) {
double denom = 10;
double ret = 0;
for (size_t i = 0; i < sizeof(t.v) / sizeof(t.v[0]); ++i) {
ret += denom * t.v[i];
denom /= 100;
}
return ret * pow(10, t.e - 1);
}
void num_println(struct num_s t) {
printf("%f =", num_get(t));
for (size_t i = 0; i < sizeof(t.v) / sizeof(t.v[0]); ++i) {
printf(" %d", t.v[i]);
}
printf(" %d\n", t.e);
}
// returns the precision of numbers
// the smallest number we can represent in num object
double num_precision(void) {
return pow(0.1, (NUM_VALSCNT - 1) * 2) * 10;
}
int num_unittests(void) {
const double tests[][3] = {
{ 123.49, 123.5, 123.51, }
};
for (size_t i = 0; i < sizeof(tests) / sizeof(tests[0]); ++i) {
const double tmp = num_get(num_create(tests[i][1]));
if (!(tests[i][0] <= tmp && tmp <= tests[i][2])) {
return i + 1;
}
}
return 0;
}
int main() {
num_println(num_create(12.3456789));
num_println(num_create(123.5));
num_println(num_create(12.35));
printf("%d\n", num_unittests());
return 0;
}

Related

Arithmetic Coding FPAQ0 (a simple order-0 arithmetic file compressor )

I am trying to understand the code of fpaq0 aritmetic compressor but I am not able to fully understand it.Here is the link to the code -fpaq0.cpp
I am not able to understand exactly the how ct[512]['2] and cxt are working.Also I am not very much clear how decoder is working.Why before encoding every charater e.encode(0) is being called.
NOTE; I have understood the arithmetic coder presented in the link-Data Compression with Arithmetic Encoding
void update(int y) {
if (++ct[cxt][y] > 65534) {
ct[cxt][0] >>= 1;
ct[cxt][1] >>= 1;
}
if ((cxt+=cxt+y) >= 512)
cxt=1;
}
// Assume a stationary order 0 stream of 9-bit symbols
int p() const {
return 4096*(ct[cxt][1]+1)/(ct[cxt][0]+ct[cxt][1]+2);
}
inline void Encoder::encode(int y) {
// Update the range
const U32 xmid = x1 + ((x2-x1) >> 12) * predictor.p();
assert(xmid >= x1 && xmid < x2);
if (y)
x2=xmid;
else
x1=xmid+1;
predictor.update(y);
// Shift equal MSB's out
while (((x1^x2)&0xff000000)==0) {
putc(x2>>24, archive);
x1<<=8;
x2=(x2<<8)+255;
}
}
inline int Encoder::decode() {
// Update the range
const U32 xmid = x1 + ((x2-x1) >> 12) * predictor.p();
assert(xmid >= x1 && xmid < x2);
int y=0;
if (x<=xmid) {
y=1;
x2=xmid;
}
else
x1=xmid+1;
predictor.update(y);
// Shift equal MSB's out
while (((x1^x2)&0xff000000)==0) {
x1<<=8;
x2=(x2<<8)+255;
int c=getc(archive);
if (c==EOF) c=0;
x=(x<<8)+c;
}
return y;
}
fpaq0 is a file compressor which uses an order-0 bitwise model for modeling and uses 12-bits carry-less arithmetic coder for entropy coding stage. ct[512][2] stores counters for each contexts to compute symbol probabilities. The context (order-0 in fpaq0) is calculated with partial bits with a leading one (to simplify calculations).
For more easy explanation, let's skip EOF symbol for now. Order-0 context calculated as follow without EOF symbol (simplified):
// Full byte encoding
int cxt = 1; // context starts with leading one
for (int i = 0; i < 8; ++i) {
// Encoding part
int y = ReadNextBit();
int p = GetProbability(ctx);
EncodeBit(y, p);
// Model updating
UpdateCounter(cxt, y); // Update related counter
cxt = (cxt << 1) | y; // shift left and insert new bit
}
For decoding, context is used without EOF symbol like following (simplified):
// Full byte decoding
int cxt = 1; // context starts with leading one
for (int i = 0; i < 8; ++i) {
// Decoding part
int p = GetProbability(ctx);
int y = DecodeBit(p);
WriteBit(y);
// Model updating
UpdateCounter(cxt, y); // Update related counter
cxt = (cxt << 1) | y; // shift left and insert new bit
}
fpaq0 is designed as a streaming compressor. Meaning that it doesn't need to know exact length of the input stream. So, the question how decoder should know when to stop? EOF symbol used exactly for that. While encoding every single byte, a zero bit is encoded as a flag to indicate there is more data to follow. One indicates we reached the end of stream. So, decoder knows when to stop. That's the reason why our context model is 9-bits (EOF flag + 8 bits data).
Now, the last part: probability calculation. fpaq0 uses just counts of past symbols under order-0 context to calculate final probability.
n0 = count of 0
n1 = count of 1
p = n1 / (n0 + n1)
There are two implementation details that should be addressed: counter overflow and division by zero.
Counter overflow is addressed by halving both counts when they reach a threshold. Since, we're dealing with p, it makes sense.
Division by zero is addressed by inserting one into formula for each variables. So,
p = (n1 + 1) / ((n0 + 1) + (n1 + 1))

Why am I getting "nan" values in C++?

I am making a Correlogram for an image. For each pixel, a correlogram finds the pixels of same color within a certain range of distance, d. Correlogram is a 2D matrix i.e. correlogram[color][distance]. The calculation of a Correlogram is somewhat similar to that of a Histogram.
My Code: I am posting some major part of the code in which all the calculations are going on. Rest of the code (which i didn't post) is used to fulfill other condtions and therefore is not necessary.
Problem: In my final correlogram[][] , some values are "nan". I have checked the code but i am not able to find where is the problem in my calculation/syntax.
int ColorBins = 180;
int DistanceRange = 5;
double calcCorrelogram(Mat hsvImage)
{
double correlogram[ColorBins][DistanceRange];
int pixelNum[ColorBins]; //Used to count the number of pixels of same color
Mat hsvPlanes[3];
split(hsvImage, hsvPlanes);
for(int pi=0; pi<hsvImage.rows; pi++)
{
for(int pj=0; pj<hsvImage.cols; pj++)
{
int pixelColor = (int)hsvPlanes[0].at<uchar>(pi,pj);
pixelNum[pixelColor]++;
for(int d=1; d<=DistanceRange; d++)
{
int sameColorNum=0; //* number of pixels with same color in the d-distance boundary */
int totalBoundaryNum=0; //* total number of pixels in the d-distance boundary */
for(int i= pi-d, j= pj-d; j<=pj+d; j++)
{
if(i<0)
break;
if(j<0 || j>=hsvImage.cols)
continue;
int neighbourColor = (int)hsvPlanes[0].at<uchar>(i,j);
if(pixelColor == neighbourColor)
{
sameColorNum++;
}
totalBoundaryNum++;
correlogram[pixelColor][d-1] = correlogram[pixelColor][d-1] + (double)sameColorNum / (double)totalBoundaryNum;
}
}
}
for(int c=0; c<ColorBins; c++)
{
for(int d=0; d<DistanceRange; d++)
{
if(pixelNum[c] != 0)
correlogram[c][d] = correlogram[c][d] / (double)pixelNum[c];
}
}
}
NaNs are generally created when you divide zero by zero or multiply zero by infinity. One easy way to check for abnormal numbers like NaN and infinity is to multiply by zero and check if the result is zero:
bool is_valid_double(double x)
{
return x*0.0==0.0;
}
This will return false if x is either NaN or infinity.
Then you can sprinkle your code with assertions to help find where things are going wrong:
assert(is_valid_double(correlogram[c][d]));
Once you get a crash due to an assertion failure, you can use the debugger to look at the state of the program to help determine what is going on.

Weird behaviour in a for loop changing the results

i've got a weird problem in my code.
here's the context : in my method i create an object and then i fill the (int) buffer of this object with data in TWO "for loops".
The problem is , when i insert a printf in my loop to look at the data into my buffer, it change the data in the buffer.
actually, the result in the buffer is different if there's a printf inside the loop or not
Heres's my code, maybe it can help to understand :
bool Mod::Realiser(FFTResult * inputdata,FFTSample_s * & moduleData){
bool done = true;
float module;
unsigned int r,n;
moduleData = new FFTSample_s(NbPointsSample);
unsigned int limit = NbPointsSample >> 1;
int iGain= 0;
for (n = CentrageFFT, r = 0; r < limit; n++, r++)
{
module = inputdata->buffer[n][0] * inputdata->buffer[n][0] + inputdata->buffer[n][1] * inputdata->buffer[n][1];
// printf(" M = %lf\n",module);
moduleData->buffer[r] = (int)(10.0*log10(module)) + iGain;
}
for (n = 0; n < limit; n++, r++)
{
module = inputdata->buffer[n][0] * inputdata->buffer[n][0] + inputdata->buffer[n][1] * inputdata->buffer[n][1];
moduleData->buffer[r] = (int)(10.0*log10(module)) + iGain;
}
/* for (int i=0;i<2048;i++){
printf(" X = %lf \n",inputdata->buffer[i][0]);
printf(" Y = %lf \n",inputdata->buffer[i][1]);
printf(" M = %d\n",moduleData->buffer[i]);
}*/
This is normal behavior. See What Every Computer Scientist Should Know about Floating-Point Arithmetic. By passing a floating point number to printf, you probably force the implementation to convert it into canonical float form from an internal form that happens to have higher precision.
The results can be different. There is not one and only one right answer.
Also:
"The result of a + b is stored in a temporary destination of unspecified precision. Neither the C++ or IEEE standards mandate what precision intermediate calculations are done to and this intermediate precision will affect your results. The temporary result could equally easily be stored in a float or a double and there are significant advantages to both options. " - Floating-Point Determinism

How can you convert a std::bitset<64> to a double?

Is there a way to convert a std::bitset<64> to a double without using any external library (Boost, etc.)? I am using a bitset to represent a genome in a genetic algorithm and I need a way to convert a set of bits to a double.
The C++11 road:
union Converter { uint64_t i; double d; };
double convert(std::bitset<64> const& bs) {
Converter c;
c.i = bs.to_ullong();
return c.d;
}
EDIT: As noted in the comments, we can use char* aliasing as it is unspecified instead of being undefined.
double convert(std::bitset<64> const& bs) {
static_assert(sizeof(uint64_t) == sizeof(double), "Cannot use this!");
uint64_t const u = bs.to_ullong();
double d;
// Aliases to `char*` are explicitly allowed in the Standard (and only them)
char const* cu = reinterpret_cast<char const*>(&u);
char* cd = reinterpret_cast<char*>(&d);
// Copy the bitwise representation from u to d
memcpy(cd, cu, sizeof(u));
return d;
}
C++11 is still required for to_ullong.
Most people are trying to provide answers that let you treat the bit-vector as though it directly contained an encoded int or double.
I would advise you completely avoid that approach. While it does "work" for some definition of working, it introduces hamming cliffs all over the place. You usually want your encoding to arrange things so that if two decoded values are near to one another, then their encoded values are near to one another as well. It also forces you to use 64-bits of precision.
I would manage the conversion manually. Say you have three variables to encode, x, y, and z. Your domain expertise can be used to say, for example, that -5 <= x < 5, 0 <= y < 100, and 0 <= z < 1, where you need 8 bits of precision for x, 12 bits for y, and 10 bits for z. This gives you a total search space of only 30 bits. You can have a 30 bit string, treat the first 8 as encoding x, the next 12 as y, and the last 10 as z. You are also free to gray code each one to remove the hamming cliffs.
I've personally done the following in the past:
inline void binary_encoding::encode(const vector<double>& params)
{
unsigned int start=0;
for(unsigned int param=0; param<params.size(); ++param) {
// m_bpp[i] = number of bits in encoding of parameter i
unsigned int num_bits = m_bpp[param];
// map the double onto the appropriate integer range
// m_range[i] is a pair of (min, max) values for ith parameter
pair<double,double> prange=m_range[param];
double range=prange.second-prange.first;
double max_bit_val=pow(2.0,static_cast<double>(num_bits))-1;
int int_val=static_cast<int>((params[param]-prange.first)*max_bit_val/range+0.5);
// convert the integer to binary
vector<int> result(m_bpp[param]);
for(unsigned int b=0; b<num_bits; ++b) {
result[b]=int_val%2;
int_val/=2;
}
if(m_gray) {
for(unsigned int b=0; b<num_bits-1; ++b) {
result[b]=!(result[b]==result[b+1]);
}
}
// insert the bits into the correct spot in the encoding
copy(result.begin(),result.end(),m_genotype.begin()+start);
start+=num_bits;
}
}
inline void binary_encoding::decode()
{
unsigned int start = 0;
// for each parameter
for(unsigned int param=0; param<m_bpp.size(); param++) {
unsigned int num_bits = m_bpp[param];
unsigned int intval = 0;
if(m_gray) {
// convert from gray to binary
vector<int> binary(num_bits);
binary[num_bits-1] = m_genotype[start+num_bits-1];
intval = binary[num_bits-1];
for(int i=num_bits-2; i>=0; i--) {
binary[i] = !(binary[i+1] == m_genotype[start+i]);
intval += intval + binary[i];
}
}
else {
// convert from binary encoding to integer
for(int i=num_bits-1; i>=0; i--) {
intval += intval + m_genotype[start+i];
}
}
// convert from integer to double in the appropriate range
pair<double,double> prange = m_range[param];
double range = prange.second - prange.first;
double m = range / (pow(2.0,double(num_bits)) - 1.0);
// m_phenotype is a vector<double> containing all the decoded parameters
m_phenotype[param] = m * double(intval) + prange.first;
start += num_bits;
}
}
Note that for reasons that probably don't matter to you, I wasn't using bit vectors -- just ordinary vector<int> to encoding things. And of course, there's a bunch of stuff tied into this code that isn't shown here, but you can probably get the basic idea.
One other note, if you're doing GPU calculations or if you have a particular problem such that 64 bits are the appropriate size anyway, it may be worth the extra overhead to stuff everything into native words. Otherwise, I would guess that the overhead you add to the search process will probably overwhelm whatever benefits you get by faster encoding and decoding.
Edit:: I've decided that I was being a bit silly with this. While you do end up with a double it assumes that the bitset holds an integer... which is a big assumption to make. You will end up with a predictable and repeatable value per bitset but still I don't think that this is what the author intended.
Well if you iterate over the bit values and do
output_double += pow( 2, 64-(bit_position+1) ) * bit_value;
That would work. As long as it is big-endian

Arduino mega queue

I wrote this simple code which reads a length from the Sharp infrared sensor, end presents the average meter in cm (unit) by serial.
When write this code for the Arduino Mega board, the Arduino starts a blinking LED (pin 13) and the program does nothing. Where is the bug in this code?
#include <QueueList.h>
const int ANALOG_SHARP = 0; //Set pin data from sharp.
QueueList <float> queuea;
float cm;
float qu1;
float qu2;
float qu3;
float qu4;
float qu5;
void setup() {
Serial.begin(9600);
}
void loop() {
cm = read_gp2d12_range(ANALOG_SHARP); //Convert to cm (unit).
queuea.push(cm); //Add item to queue, when I add only this line Arduino crash.
if ( 5 <= queuea.peek()) {
Serial.println(average());
}
}
float read_gp2d12_range(byte pin) { //Function converting to cm (unit).
int tmp;
tmp = analogRead(pin);
if (tmp < 3)
return -1; // Invalid value.
return (6787.0 /((float)tmp - 3.0)) - 4.0;
}
float average() { //Calculate average length
qu1 += queuea.pop();
qu2 += queuea.pop();
qu3 += queuea.pop();
qu4 += queuea.pop();
qu5 += queuea.pop();
float aver = ((qu1+qu2+qu3+qu4+qu5)/5);
return aver;
}
I agree with the peek() -> count() error listed by vhallac. But I'll also point out that you should consider averaging by powers of 2 unless there is a strong case to do otherwise.
The reason is that on microcontrollers, division is slow. By averaging over a power of 2 (2,4,8,16,etc.) you can simply calculate the sum and then bitshift it.
To calculate the average of 2: (v1 + v2) >> 1
To calculate the average of 4: (v1 + v2 + v3 + v4) >> 2
To calculate the average of n values (where n is a power of 2) just right bitshift the sum right by [log2(n)].
As long as the datatype for your sum variable is big enough and won't overflow, this is much easier and much faster.
Note: this won't work for floats in general. In fact, microcontrollers aren't optimized for floats. You should consider converting from int (what I'm assuming you're ADC is reading) to float at the end after the averaging rather than before.
By converting from int to float and then averaging floats you are losing more precision than averaging ints than converting the int to a float.
Other:
You're using the += operator without initializing the variables (qu1, qu2, etc.) -- it's good practice to initialize them if you're going to use += but it looks as if = would work fine.
For floats, I'd have written the average function as:
float average(QueueList<float> & q, int n)
{
float sum = 0;
for(int i=0; i<n; i++)
{
sum += q.pop();
}
return (sum / (float) n);
}
And called it: average(queuea, 5);
You could use this to average any number of sensor readings and later use the same code to later average floats in a completely different QueueList. Passing the number of readings to average as a parameter will really come in handy in the case that you need to tweak it.
TL;DR:
Here's how I would have done it:
#include <QueueList.h>
const int ANALOG_SHARP=0; // set pin data from sharp
const int AvgPower = 2; // 1 for 2 readings, 2 for 4 readings, 3 for 8, etc.
const int AvgCount = pow(2,AvgPow);
QueueList <int> SensorReadings;
void setup(){
Serial.begin(9600);
}
void loop()
{
int reading = analogRead(ANALOG_SHARP);
SensorReadings.push(reading);
if(SensorReadings.count() > AvgCount)
{
int avg = average2(SensorReadings, AvgPower);
Serial.println(gpd12_to_cm(avg));
}
}
float gp2d12_to_cm(int reading)
{
if(reading <= 3){ return -1; }
return((6787.0 /((float)reading - 3.0)) - 4.0);
}
int average2(QueueList<int> & q, int AvgPower)
{
int AvgCount = pow(2, AvgPower);
long sum = 0;
for(int i=0; i<AvgCount; i++)
{
sum += q.pop();
}
return (sum >> AvgPower);
}
You are using queuea.peek() to obtain the count. This will only return the last element in queue. You should use queuea.count() instead.
Also you might consider changing the condition tmp < 3 to tmp <= 3. If tmp is 3, you divide by zero.
Great improvement jedwards, however the first question I have is why use queuelist instead of an int array.
As an example I would do the following:
int average(int analog_reading)
{
#define NUM_OF_AVG 5
static int readings[NUM_OF_AVG];
static int next_position;
static int sum;
if (++next_position >= NUM_OF_AVG)
{
next_position=0;
}
reading[next_position]=analog_reading;
for(int i=0; i<NUM_OF_AVG; i++)
{
sum += reading[i];
}
average = sum/NUM_OF_AVG
}
Now I compute a new rolling average with every reading and it eliminates all the issues related to dynamic memory allocation (memory fragmentation, no available memory, memory leaks) in a embedded device.
I appreciate and understand the use of shifting for a division by 2,4 or 8, however I would stay away from that technique for two reasons.
I think readability and maintainability of the source code is more important then saving a little bit of time with a shift instead of a divide unless you can test and verify the divide is a bottleneck.
Second, I believe most current optimizing compilers will do a shift if possible, I know GCC does.
I will leave refactoring out the for loop for the next guy.