Floating point rounding errors

Computers store floating point numbers in IEEE-754 format. This is imprecise and can result in rounding errors after the 8th or 9th decimal place. The following example shows the error in a simplified manner:

perl -E 'say 0.1 + 0.2 == 0.3 ? "true" : "false"'

python -c 'print 0.1 + 0.2 == 0.3'

ruby -e 'puts 0.1 + 0.2 == 0.3'

php -r 'print 0.1 + 0.2 == 0.3 ? "true" : "false";'

All of these print out false which is obviously the incorrect answer. Most languages recommend some type of rounding or comparison library when comparing floating point values for this reason.

This rounding error is present in JavaScript too:

console.log(0.1 + 0.2 == 0.3 ? "True" : "False");
Leave A Reply
All content licensed under the Creative Commons License