[sf-lug] Old Calculator Problems

[Sujit K M]

On Thu, Aug 11, 2016 at 8:02 PM, Sujit K M <kmsujit at gmail.com> wrote:
> On Thu, Aug 11, 2016 at 10:19 AM, Sujit K M <kmsujit at gmail.com> wrote:
>> I found something wron in your stackover link. Python lets you round
>> to 10 pow -15.
>
> Yeah if you add 0.1 with any number there is an result in these terms.

I ran the following computation below on a python shell.

>>> 0.2 + 0.1
0.30000000000000004
>>> x = 0.2 + 0.1
>>> x
0.30000000000000004
>>> x = 0.2 + 0.1 + 1
>>> x
1.3
>>> y = 1
>>> x -1
0.30000000000000004
>>> x = x - y
>>> x
0.30000000000000004
>>> x = 1.1 - 1
>>> x
0.10000000000000009
>>> x = 1.2 - 1
>>> x
0.19999999999999996

Floats are being stored in double precision format in and in the case of
Values greater than 1.0 it is being stored in normalized form. I don't know
what normalization is being used in Python but should be using normalization
of some form so that sums are normalized greater than 1.0 and arithmetic
computation carried out. Below is a link I found Useful.

http://www.cs.uwm.edu/~cs151/Bacon/Lecture/HTML/ch03s10.html