Opened 16 years ago

Closed 14 years ago

Last modified 13 years ago

#11206 closed (fixed)

The floatformat template tag doesn't work with a value of 0 and a precision of 7 or higher.

Reported by: Tai Lee Owned by: nobody
Component: Template system Version: dev
Severity: Keywords: floatformat precision 7 decimal sprintdec2010
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

>>> floatformat(0.123456789, 6)
u'0.123457'
>>> floatformat(0.123456789, 7)
u'0.1234568'
>>> floatformat(0.0, 6)
u'0.000000'
>>> floatformat(0.0, 7)
u'0E-7'

Looks like the Decimal module (which floatformat uses to perform accurate rounding) is causing trouble.

>>> Decimal('0.123456789').quantize(Decimal('1.000000'))
Decimal("0.123457")
>>> Decimal('0.123456789').quantize(Decimal('1.0000000'))
Decimal("0.1234568")
>>> Decimal('0.0').quantize(Decimal('1.000000'))
Decimal("0.000000")
>>> Decimal('0.0').quantize(Decimal('1.0000000'))
Decimal("0E-7")

Attachments (3)

11206-floatformat-r10837.diff (778 bytes ) - added by Tai Lee 16 years ago.
Failing test case.
11206-floatformat-r14785.diff (2.4 KB ) - added by Tai Lee 14 years ago.
11206-floatformat-r14785.2.diff (2.5 KB ) - added by Tai Lee 14 years ago.

Download all attachments as: .zip

Change History (15)

by Tai Lee, 16 years ago

Failing test case.

comment:1 by Facundo Batista, 16 years ago

Note that the behaviour of Decimal module in that case is correct.

The Decimal Specification [0], in the "to-scientific-string – conversion to numeric string" section, says:

"""
If the exponent is less than or equal to zero and the adjusted exponent is greater than or equal to -
6, the number will be converted to a character form without using exponential notation.
...
Otherwise (that is, if the exponent is positive, or the adjusted exponent is less than -6), the number
will be converted to a character form using exponential notation.
"""

[0] http://speleotrove.com/decimal/

comment:2 by Alex Gaynor, 15 years ago

Triage Stage: UnreviewedDesign decision needed

comment:3 by Carl Meyer, 14 years ago

Triage Stage: Design decision neededAccepted

I'm going to move this to Accepted, on the basis that even though the current behavior follows Decimal and may be correct according to some standard, I highly doubt that it would ever be the desired behavior in a real usage of floatformat. I think it's pretty low priority, though, and would need to see a working patch without any obvious major downsides (including the implementation being hugely complex and potentially difficult to maintain).

I don't feel strongly about this, and would be pretty open to an argument for wontfix.

by Tai Lee, 14 years ago

comment:4 by Tai Lee, 14 years ago

Has patch: set
Keywords: sprintdec2010 added
milestone: 1.3

Just uploaded a patch that fixes this with tests. It reconstructs the number from the DecimalTuple object to avoid conversion into scientific format. I don't think the implementation is too complex or difficult to maintain. I don't think docs are needed as this is a fix to unexpected behaviour.

by Tai Lee, 14 years ago

comment:5 by Tai Lee, 14 years ago

Updated the patch to use a list comprehension and consistent unicode strings as per brodie's feedback in #django-sprint.

comment:6 by Adam Nelson, 14 years ago

It seems like the simpler solution is to use .normalize(). I could be wrong though - I haven't gone through all the possible decimal/float issues

>>> Decimal('0.0').quantize(Decimal('1.000000000'))
Decimal('0E-9')
>>> Decimal('0.0').quantize(Decimal('1.000000000')).normalize()
Decimal('0')

comment:7 by Tai Lee, 14 years ago

Decimal.normalize() strips trailing zeroes (and the decimal point), and still converts some numbers to scientific notation.

>>> decimal.Decimal('0.0000001').normalize()
Decimal('1E-7')

comment:8 by Adam Vandenberg, 14 years ago

As a possibly unrelated side note, dealing with Decimal values in the external MS SQL Server backend gives us no end of grief: http://code.google.com/p/django-mssql/source/browse/sqlserver_ado/dbapi.py?r=2451845d9bf8c77746f569d10dae417d26d6c24f#179

We end up using "as_tuple" and counting string lengths.

comment:9 by Ramiro Morales, 14 years ago

Triage Stage: AcceptedReady for checkin

Patch looks correct, Decimal-related code modification got positive review by Facundo Batista (Decimal module co-maintainter).

comment:10 by Ramiro Morales, 14 years ago

Resolution: fixed
Status: newclosed

In [15736]:

Fixed #11206 -- Ensure that the floatformat template filter doesn't switch to scientific notation when asked to format a zero value with more than six decimal places. Thanks Tai Lee for the report and fix and Facundo Batista for his help when Decimal module expertise was needed.

comment:11 by Ramiro Morales, 14 years ago

In [15738]:

[1.2.X] Fixed #11206 -- Ensure that the floatformat template filter doesn't switch to scientific notation when asked to format a zero value with more than six decimal places. Thanks Tai Lee for the report and fix and Facundo Batista for his help when Decimal module expertise was needed.

Backport of [15736] from trunk

comment:12 by Jacob, 13 years ago

milestone: 1.3

Milestone 1.3 deleted

Note: See TracTickets for help on using tickets.
Back to Top