Saturday, June 2, 2012

Round and round around the floats

Isn't rounding numbers easy? Well, not always. And not always consistent:

Python (the same for C/C++,):

>>> "%.1f" % 0.05
'0.1'
>>> "%.1f" % 0.15
'0.1'
>>> "%.1f" % 0.25
'0.2'
>>> "%.1f" % 0.35
'0.3'
>>> "%.1f" % 0.45
'0.5'
>>> "%.1f" % 0.55
'0.6'
>>> "%.1f" % 0.65
'0.7'
>>> "%.1f" % 0.75
'0.8'
>>> "%.1f" % 0.85
'0.8'
>>> "%.1f" % 0.95
'0.9'

But that is nothing compared to results obtained from MySQL today:



+--------------+----------------+
| sum(penalty) | count(penalty) |
+--------------+----------------+
|         5091 |             24 |
+--------------+----------------+


+------------------------+--------------+
| ROUND(avg(penalty), 2) | avg(penalty) |
+------------------------+--------------+
|                 212.12 |      212.125 |
+------------------------+--------------+

+-------------------+-------------------+
| ROUND(212.125, 2) | ROUND(5091/24, 2) |
+-------------------+-------------------+
|            212.13 |            212.13 |
+-------------------+-------------------+








No comments:

Post a Comment