[sf-lug] Old Calculator Problems

[Asheesh Laroia]

Hi Sujit,

Very likely you are running into the inaccuracy of so-called "floating
point" arithmetic. The topic is fascinating and mind-boggling, in my
opinion.

These are interesting references in my opinion:

- "Why do simple math operations on floating point return unexpected
(inaccurate) results in VB.Net and Python?"
http://stackoverflow.com/questions/1594985/why-do-simple-math-operations-on-floating-point-return-unexpected-inaccurate-r

- "What you never wanted to know about floating point but will be forced to
find out" http://www.volkerschatz.com/science/float.html

- https://en.wikipedia.org/wiki/IEEE_floating_point

Cheers,

Asheesh.



On Wed, Aug 10, 2016 at 8:34 AM, Sujit K M <kmsujit at gmail.com> wrote:

> Hi All,
>
> Just inspired by the Shell Scripting sciences I got inspired by something
> else.
>
> I find something strange with my calculator it calculates from
> the width of the decimal on my desktop.
>
> For Example
>           1.  6*2/3 = 4
>           2.  6*0.667 = 4.002
>           3.  6*0.6666666667 =   4.0000000002
>           4.  6*0.6666666666666667 = 4.
>
> Are there any standards for this. #4 should also lead to a value of
> 4.0000000000000002.
> Why is the calculator behaving strangely? There is no way in the world
> width of the decimal
> is affecting the result. Its a Desktop PC Calculator(Sounds Microsoft).
>
> What is the problem? I would love to dwell into the sciences that go
> into, might be an
> opensource calculator with source code ofcourse.
>
> Regards,
> Sujit K M
>
> _______________________________________________
> sf-lug mailing list
> sf-lug at linuxmafia.com
> http://linuxmafia.com/mailman/listinfo/sf-lug
> Information about SF-LUG is at http://www.sf-lug.org/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://linuxmafia.com/pipermail/sf-lug/attachments/20160810/452e4c32/attachment.html>