Code

Opened 15 months ago

Closed 15 months ago

Last modified 15 months ago

#20251 closed Bug (invalid)

UnicodeEncodeError with DecimalField

Reported by: anonymous Owned by: nobody
Component: Forms Version: 1.5
Severity: Normal Keywords:
Cc: bmispelon@… Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

My form:

class TestForm(forms.Form):
    value = forms.DecimalField()

When enteing "1 грн"

'decimal' codec can't encode characters in position 4-6: invalid decimal Unicode string

Method to_python catch only 'DecimalException' but not 'UnicodeEncodeError'.


C:\Python\lib\site-packages\django\forms\fields.py in to_python
            """
            if value in validators.EMPTY_VALUES:
                return None
            if self.localize:
                value = formats.sanitize_separators(value)
            value = smart_text(value).strip()
            try:
                value = Decimal(value)
    ...
            except DecimalException:
                raise ValidationError(self.error_messages['invalid'])

            return value

        def validate(self, value):

            super(DecimalField, self).validate(value)


Attachments (0)

Change History (7)

comment:1 Changed 15 months ago by bmispelon

  • Cc bmispelon@… added
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Resolution set to invalid
  • Status changed from new to closed

Hi,

Could you provide more details on how to reproduce the issue?

I tried the following (with python 2.7 and 3.2) but could not trigger the UnicodeEncodeError:

class TestForm(forms.Form):
    field = forms.DecimalField()

data = {'field': u"1 грн"}
form = TestForm(data)
if form.is_valid():
    print(form.cleaned_data)
else:
    print('Invalid form')

This prints out "Invalid form", without triggering any other error.

I also simply tried Decimal("1 грн") but this raise a DecimalException, not a UnicodeEncodeError.

comment:2 Changed 15 months ago by anonymous

#!/usr/bin/python
# -*- coding: utf-8 -*-
import sys
import os

os.environ['DJANGO_SETTINGS_MODULE'] = 'agu.settings'

from agu import settings
from django import forms

class TestForm(forms.Form):
    field = forms.DecimalField()

data = {'field': u"1 грн"}
form = TestForm(data)
if form.is_valid():
    print(form.cleaned_data)
else:
    print('Invalid form')

Runing on Windows7 64bit, python 2.7, Django 1.5

Traceback (most recent call last):
  File "temp.py", line 19, in <module>
    if form.is_valid():
  File "C:\Python\lib\site-packages\django\forms\forms.py", line 126, in is_valid
    return self.is_bound and not bool(self.errors)
  File "C:\Python\lib\site-packages\django\forms\forms.py", line 117, in _get_errors
    self.full_clean()
  File "C:\Python\lib\site-packages\django\forms\forms.py", line 272, in full_clean
    self._clean_fields()
  File "C:\Python\lib\site-packages\django\forms\forms.py", line 287, in _clean_fields
    value = field.clean(value)
  File "C:\Python\lib\site-packages\django\forms\fields.py", line 154, in clean
    value = self.to_python(value)
  File "C:\Python\lib\site-packages\django\forms\fields.py", line 293, in to_python
    value = Decimal(value)
UnicodeEncodeError: 'decimal' codec can't encode characters in position 2-4: invalid decimal Unicode string

comment:3 Changed 15 months ago by bmispelon

  • Resolution invalid deleted
  • Status changed from closed to new

Interesting.

Does Decimal(u"1 грн") also trigger a UnicodeEncodeError in a plain python shell?

I'm reopening because I don't have access to a windows installation to test this.

comment:4 Changed 15 months ago by anonymous

Looks like it's something with the my options in settings.py
Clear django-project work's fine. I will test this in detail.

comment:5 Changed 15 months ago by anonymous

  • Resolution set to needsinfo
  • Status changed from new to closed

comment:6 Changed 15 months ago by anonymous

I use cdecimal for speedup operations, it problem

import cdecimal # speedup decimal
sys.modules["decimal"] = cdecimal

comment:7 Changed 15 months ago by bmispelon

  • Resolution changed from needsinfo to invalid

OK, that explains it. It looks like you've stumbled into a bug of cdecimal.

FYI, python 3.3 integrated cdecimal with some further performance improvements so if you have a chance, you should check it out.

I'm marking this as invalid since the bug is not in django itself. If you disagree, feel free to re-open the ticket or start a thread on the django-dev mailing list.

Thanks.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.