Opened 6 years ago

Closed 4 years ago

Last modified 4 years ago

#11206 closed (fixed)

The floatformat template tag doesn't work with a value of 0 and a precision of 7 or higher.

Reported by: mrmachine Owned by: nobody
Component: Template system Version: master
Severity: Keywords: floatformat precision 7 decimal sprintdec2010
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

>>> floatformat(0.123456789, 6)
u'0.123457'
>>> floatformat(0.123456789, 7)
u'0.1234568'
>>> floatformat(0.0, 6)
u'0.000000'
>>> floatformat(0.0, 7)
u'0E-7'

Looks like the Decimal module (which floatformat uses to perform accurate rounding) is causing trouble.

>>> Decimal('0.123456789').quantize(Decimal('1.000000'))
Decimal("0.123457")
>>> Decimal('0.123456789').quantize(Decimal('1.0000000'))
Decimal("0.1234568")
>>> Decimal('0.0').quantize(Decimal('1.000000'))
Decimal("0.000000")
>>> Decimal('0.0').quantize(Decimal('1.0000000'))
Decimal("0E-7")

Attachments (3)

11206-floatformat-r10837.diff (778 bytes) - added by mrmachine 6 years ago.
Failing test case.
11206-floatformat-r14785.diff (2.4 KB) - added by mrmachine 4 years ago.
11206-floatformat-r14785.2.diff (2.5 KB) - added by mrmachine 4 years ago.

Download all attachments as: .zip

Change History (15)

Changed 6 years ago by mrmachine

Failing test case.

comment:1 Changed 6 years ago by facundobatista

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

Note that the behaviour of Decimal module in that case is correct.

The Decimal Specification [0], in the "to-scientific-string – conversion to numeric string" section, says:

"""
If the exponent is less than or equal to zero and the adjusted exponent is greater than or equal to -
6, the number will be converted to a character form without using exponential notation.
...
Otherwise (that is, if the exponent is positive, or the adjusted exponent is less than -6), the number
will be converted to a character form using exponential notation.
"""

[0] http://speleotrove.com/decimal/

comment:2 Changed 6 years ago by Alex

  • Triage Stage changed from Unreviewed to Design decision needed

comment:3 Changed 5 years ago by carljm

  • Triage Stage changed from Design decision needed to Accepted

I'm going to move this to Accepted, on the basis that even though the current behavior follows Decimal and may be correct according to some standard, I highly doubt that it would ever be the desired behavior in a real usage of floatformat. I think it's pretty low priority, though, and would need to see a working patch without any obvious major downsides (including the implementation being hugely complex and potentially difficult to maintain).

I don't feel strongly about this, and would be pretty open to an argument for wontfix.

Changed 4 years ago by mrmachine

comment:4 Changed 4 years ago by mrmachine

  • Has patch set
  • Keywords sprintdec2010 added
  • milestone set to 1.3

Just uploaded a patch that fixes this with tests. It reconstructs the number from the DecimalTuple object to avoid conversion into scientific format. I don't think the implementation is too complex or difficult to maintain. I don't think docs are needed as this is a fix to unexpected behaviour.

Changed 4 years ago by mrmachine

comment:5 Changed 4 years ago by mrmachine

Updated the patch to use a list comprehension and consistent unicode strings as per brodie's feedback in #django-sprint.

comment:6 Changed 4 years ago by adamnelson

It seems like the simpler solution is to use .normalize(). I could be wrong though - I haven't gone through all the possible decimal/float issues

>>> Decimal('0.0').quantize(Decimal('1.000000000'))
Decimal('0E-9')
>>> Decimal('0.0').quantize(Decimal('1.000000000')).normalize()
Decimal('0')

comment:7 Changed 4 years ago by mrmachine

Decimal.normalize() strips trailing zeroes (and the decimal point), and still converts some numbers to scientific notation.

>>> decimal.Decimal('0.0000001').normalize()
Decimal('1E-7')

comment:8 Changed 4 years ago by adamv

As a possibly unrelated side note, dealing with Decimal values in the external MS SQL Server backend gives us no end of grief: http://code.google.com/p/django-mssql/source/browse/sqlserver_ado/dbapi.py?r=2451845d9bf8c77746f569d10dae417d26d6c24f#179

We end up using "as_tuple" and counting string lengths.

comment:9 Changed 4 years ago by ramiro

  • Triage Stage changed from Accepted to Ready for checkin

Patch looks correct, Decimal-related code modification got positive review by Facundo Batista (Decimal module co-maintainter).

comment:10 Changed 4 years ago by ramiro

  • Resolution set to fixed
  • Status changed from new to closed

In [15736]:

Fixed #11206 -- Ensure that the floatformat template filter doesn't switch to scientific notation when asked to format a zero value with more than six decimal places. Thanks Tai Lee for the report and fix and Facundo Batista for his help when Decimal module expertise was needed.

comment:11 Changed 4 years ago by ramiro

In [15738]:

[1.2.X] Fixed #11206 -- Ensure that the floatformat template filter doesn't switch to scientific notation when asked to format a zero value with more than six decimal places. Thanks Tai Lee for the report and fix and Facundo Batista for his help when Decimal module expertise was needed.

Backport of [15736] from trunk

comment:12 Changed 4 years ago by jacob

  • milestone 1.3 deleted

Milestone 1.3 deleted

Note: See TracTickets for help on using tickets.
Back to Top