Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#27394 closed Bug (fixed)

template "floatformat" tag throws ValueError for big numbers in Python 3

Reported by: mateuszf Owned by: Mariusz Felisiak
Component: Template system Version: 1.10
Severity: Normal Keywords: floatformat python3
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I'm porting application from Python 2 to Python 3.
We receive and display floating point values from external system. They may be big values.

Following example works on Python 2 with Django 1.10.2 but throws "ValueError: valid range for prec is [1, MAX_PREC]" on Python 3 with the same Django version.

def my_view(request):
    return render(request, my_template.html', { 'value':-1.323297138040798e+35 })
{{ value|floatformat:2 }}

We have separate custom tag for displaying values in scientific format but how am I supposed to check if value will be printed in scientific format before printing it in template ? This was working automatically in Python 2.

Attachments (2)

27394-test.diff (623 bytes ) - added by Tim Graham 7 years ago.
27394-test.2.diff (658 bytes ) - added by Mariusz Felisiak 7 years ago.
Test

Download all attachments as: .zip

Change History (19)

by Tim Graham, 7 years ago

Attachment: 27394-test.diff added

comment:1 by Tim Graham, 7 years ago

Triage Stage: UnreviewedAccepted
Type: UncategorizedBug

A regression test is attached, though it doesn't work on Python 2 either, returning the original value rather than the formatted version.

comment:2 by Mariusz Felisiak, 7 years ago

Owner: changed from nobody to Mariusz Felisiak
Status: newassigned

by Mariusz Felisiak, 7 years ago

Attachment: 27394-test.2.diff added

Test

in reply to:  1 comment:3 by Mariusz Felisiak, 7 years ago

Replying to Tim Graham:

A regression test is attached, though it doesn't work on Python 2 either, returning the original value rather than the formatted version.

I attached correct regression test. Previous was little bit wrong because

-1.323297138040798e+35 = -1.323297138040798 * 1035 = -132329713804079800000000000000000000

PR in progress.

Last edited 7 years ago by Mariusz Felisiak (previous) (diff)

comment:4 by Mariusz Felisiak, 7 years ago

Has patch: set

comment:5 by mateuszf, 7 years ago

I've looked at the PR and the test is:

self.assertEqual(floatformat(-1.323297138040798e+35, 2), '-132329713804079800000000000000000000.00')

But I've just retested it on Python 2 and my example is being rendered as: -1.32329713804e+35. So in Python 2 "floatformat" does not force big numbers to standard notation, and I think this tag should behave the same for Python 2 and Python 3.

From my project's perspective: values we receive from external system may be big or may be small but we always have some user defined precision. If "floatformat" will always force standard notation even for really big numbers it will become useless for us, because we don't want to print values in a way that will occupy 1/2 screen width. It's also unreadable this way.

comment:6 by Tim Graham, 7 years ago

Right, it's debatable whether or not the behavior on Python 2 should be considered a bug. The current behavior of leaving scientific notation untouched isn't documented. I don't have a strong opinion.

comment:7 by mateuszf, 7 years ago

I understand that current behavior on Python 2 may be debatable. Maybe there should be 2 versions of "floatformat" with different names, one that will always force standard notation and one that will always print something readable.
We can always implement it manually in our project if the decision will be to leave it as implemented in PR.

in reply to:  5 ; comment:8 by Mariusz Felisiak, 7 years ago

Replying to mateuszf:

I've looked at the PR and the test is:

self.assertEqual(floatformat(-1.323297138040798e+35, 2), '-132329713804079800000000000000000000.00')

But I've just retested it on Python 2 and my example is being rendered as: -1.32329713804e+35. So in Python 2 "floatformat" does not force big numbers to standard notation, and I think this tag should behave the same for Python 2 and Python 3.

Are you sure? This test passed on Python 2 and Python 3, hence in both cases floatformat returns '-132329713804079800000000000000000000.00' ie. number in a standard notation (without losing precision).

comment:9 by Tim Graham, 7 years ago

The point is that the current behavior on Python 2 leaves numbers in scientific notation untouched. Absent another argument, I don't mind keeping that behavior. I guess there's little reason to break backwards-compatibility and using floatformat with scientific notation to add a bunch of zeros doesn't seem particularly useful.

in reply to:  8 comment:10 by mateuszf, 7 years ago

Replying to felixxm:

Are you sure? This test passed on Python 2 and Python 3, hence in both cases floatformat returns '-132329713804079800000000000000000000.00' ie. number in a standard notation (without losing precision).

Well that's strange. My result was pasted directly from browser window running simple test project using Python 2.7.8 (32-bit) on Windows 7 (64-bit).
I've made some more tests reinstalling Python to Python 2.7.12 (32-bit) and Python 2.7.12 (64-bit) using the same test project and the same Django 1.10.2 and result is still the same: -1.32329713804e+35

comment:11 by Tim Graham, 7 years ago

felixxm, do you agree to keep the current behavior:

>>> floatformat(-1.323297138040798e+35, 2)
u'-1.32329713804e+35'

or do you feel there's value in changing it to output all the zeros (as done by the current PR)?

in reply to:  11 comment:12 by Mariusz Felisiak, 7 years ago

Replying to Tim Graham:

felixxm, do you agree to keep the current behavior:

>>> floatformat(-1.323297138040798e+35, 2)
u'-1.32329713804e+35'

or do you feel there's value in changing it to output all the zeros (as done by the current PR)?

It's really hard to say. The current behavior works only in python2. If we want to leave numbers in scientific notation untouched then it should be fixed for python3 and documented (I can prepare PR).

Moreover this change should be done, because without it we loose precision in python2: floatformat(-1.323297138040798e+35, 2) returns u'-1.32329713804e+35' instead of u'-1.323297138040798e+35'.

comment:13 by Tim Graham, 7 years ago

Yes, that's the plan I envisioned.

in reply to:  13 comment:14 by Mariusz Felisiak, 7 years ago

I checked current floatformat behavior in different cases:

  • python 2:
>>> floatformat(-1.323297138040798e+35, -2)
u'-132329713804000000000000000000000000'
>>> floatformat(-1.323297138040798e+35, 2)
u'-1.32329713804e+35'
>>> floatformat(1.5e-15, 20)
u'0.00000000000000150000'
>>> floatformat(1.5e-15, -20)
u'0.00000000000000150000'

after PR (in current form):

>>> floatformat(-1.323297138040798e+35, -2)
u'-132329713804079800000000000000000000'
>>> floatformat(-1.323297138040798e+35, 2)
u'-132329713804079800000000000000000000.00'
>>> floatformat(1.5e-15, 20)
u'0.00000000000000150000'
>>> floatformat(1.5e-15, -20)
u'0.00000000000000150000'
  • python 3:
>>> from django.template.defaultfilters import floatformat
>>> floatformat(-1.323297138040798e+35, -2)
'-132329713804079800000000000000000000'
>>> floatformat(-1.323297138040798e+35, 2)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/xxxyyy/lib/python3.4/site-packages/django/template/defaultfilters.py", line 173, in floatformat
    sign, digits, exponent = d.quantize(exp, ROUND_HALF_UP, Context(prec=prec)).as_tuple()
  File "/usr/lib/python3.4/decimal.py", line 3904, in __init__
    self.prec = prec if prec is not None else dc.prec
  File "/usr/lib/python3.4/decimal.py", line 3957, in __setattr__
    return self._set_integer_check(name, value, 1, 'inf')
  File "/usr/lib/python3.4/decimal.py", line 3938, in _set_integer_check
    raise ValueError("%s must be in [%d, %s]. got: %s" % (name, vmin, vmax, value))
ValueError: prec must be in [1, inf]. got: -1
>>> floatformat(1.5e-15, 20)
'0.00000000000000150000'
>>> floatformat(1.5e-15, -20)
'0.00000000000000150000'

after PR (in current form):

>>> floatformat(-1.323297138040798e+35, -2)
'-132329713804079800000000000000000000'
>>> floatformat(-1.323297138040798e+35, 2)
'-132329713804079800000000000000000000.00'
>>> floatformat(1.5e-15, 20)
'0.00000000000000150000'
>>> floatformat(1.5e-15, -20)
'0.00000000000000150000'

hence maybe it should be done like in the current PR.

comment:15 by Tim Graham, 7 years ago

Based on those results, I agree the current PR probably makes the most sense.

comment:16 by Tim Graham <timograham@…>, 7 years ago

Resolution: fixed
Status: assignedclosed

In 12f7928:

Fixed #27394 -- Added scientific notation support for big integers in floatformat filter.

comment:17 by Tim Graham <timograham@…>, 7 years ago

In 41759c90:

Refs #27394 -- Removed @expectedFailure from a floatformat test.

Note: See TracTickets for help on using tickets.
Back to Top