I am using FORTRAN 95 using Silverfrost Plato where I am trying to make real double precision variables. Currently I use 'double precision' which gives accuracy up to 12 decimal digits. But I wanted to know if we could have digits accurate up to 16 digits. Interestingly, when I tried using 'real(kind=3)' there were some residual values showing up in the last few digits that ended up making the results go wrong.
I am unable to find a proper solution to this. Any help or links to relevant documentation would be greatly appreciated.
Related
Glad to ask a question here again after more than 10 years (last one was about BASH scripting, now as I'm in corporate, guess what... it's about excel ;) )
here it's my question/issue:
I am importing data with powerquery for further analysis
I have discovered is that the values imported contains extradigits not present in the original table.
I have googled for this problem but I have not been able to find an explanation nor a solution ( a similar issue is this one this one , more than one year old, but with no feedback from Microsoft )
(columns are formatted as text in the screenshot but the issue is still present even if formatted as number)
The workaround I am using now, but I am not happy with that is the following:
I "increased decimal" to make sure all my digits are captured (in my source the entries do not have all the same significant digits),
saved as csv
imported impacted columns as number
convert columns as text (for future text match
I am really annoyed by this unwanted and unpredictable behaviour of excel.
I see a serious issue of data integrity, if we cannot rely on the powerquery/powerbi platform to maintain accurate queries, I wonder why would be use it
adding another screenshot to clarify that changing the source format to text does not solve the problem
another screenshot added following #David Bacci comments:
I think I wrongfully assumed my data was stored as text in the source, can you confirm?
If you are exporting and importing as text, then this will not happen. If you convert to number, you will lose precision. From the docs (my bold):
Represents a 64-bit (eight-byte) floating-point number. It's the most
common number type, and corresponds to numbers as you usually think of
them. Although designed to handle numbers with fractional values, it
also handles whole numbers. The Decimal Number type can handle
negative values from –1.79E +308 through –2.23E –308, 0, and positive
values from 2.23E –308 through 1.79E + 308. For example, numbers like
34, 34.01, and 34.000367063 are valid decimal numbers. The largest
precision that can be represented in a Decimal Number type is 15
digits long. The decimal separator can occur anywhere in the number.
The Decimal Number type corresponds to how Excel stores its numbers.
Note that a binary floating-point number can't represent all numbers
within its supported range with 100% accuracy. Thus, minor differences
in precision might occur when representing certain decimal numbers.
BTW, you should probably accept some of the good answers from your previous questions from 10 years ago.
I am building a neural network and using xtensor for array multiplication in feed forward. The network takes in xt::xarray<double> and outputs a decimal number between 0 and 1. I have been given a sheet for expected output. when i compare my output with the provided sheet, I found that all the results differ after exactly 7 digits. for example if the required value is 0.1234567890123456, I am getting values like 0.1234567-garbage-numbers-so-that-total-numbers-equal-16, 0.1234567993344660, 0.1234567221155667.
I know I can not get that exact number 0.1234567890123456 due to floating point math. But how can I debug/ increase precision to be close to that required number. thanks
Update:
xt::xarray<double> Layer::call(xt::xarray<double> input)
{
return xt::linalg::dot(input, this->weight) + this->bias;
}
for code I am simply calling this call method a bunch of times where weight and bias are xt::xarray<double> arrays.
I've been trying to figure this out from the Sympy docs, but I can't find any options or parameters that control precision for printed output. I don't want to change the precision of the stored numbers, I just don't need 17 digits of precision in my output. Any good way to do this?
import sympy as sp
sp.init_printing() # presumably options would go here?
a = sp.pi
sp.pprint(sp.N(a)) # I'd like to see only a few digits here, not full precision
I am new to SAS and currently working on a small piece of work with SAS.
Could I please ask what the below format means? I believe the 8. is formatting two digits to the right of the decimal place such as 896.33 but I am not sure. Not really sure what input means.
input(tablename.fieldname, 8.)
That is an INFORMAT, not a FORMAT. It means to read the first 8 characters as a number. If there is a decimal point in the data then it is used naturally. You could have up to 7 digits to the right of the decimal point (since the decimal point would use up the eighth character position). It will also support reading scientific notation so '896.33E2' would mean the number 89,633.
I'm writing a simple tool in Qt which reads data from two GPX (XML) files and combines them in a certain way. I tested my tool with track logs that contain waypoints having 6 decimal digits precision. When I read them from the GPX file, the precision gets reduced to 4 decimal digits (rounded properly). So for example this original tag:
<trkpt lat="61.510656" lon="23.777735">
turns into this when my tool writes it again:
<trkpt lat="61.5107" lon="23.7777">
Debug output shows the precision loss happens on this line:
double lat = in.attributes().value("", "lat").toString().toDouble();
but I can't see why. in is a QXmlStreamReader reading from a text file handle.
It is probably when you are writing the value back to the XML. Please post that code in your question.
If I had a guess before seeing the code, you are using QString::number to convert from the double back to a string. The default precision in the conversion is 6, which corresponds to what you are seeing. You can increase the precision to get all the decimals.