Opened 4 years ago

Closed 2 years ago

#16123 closed Bug (fixed)

DateTimeField, DateField fails covert unicode date string with non-ascii characters

Reported by: Gene Owned by: Claude Paroz <claude@…>
Component: Forms Version: 1.3
Severity: Normal Keywords: unicode, date, forms
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

it's old python bug - [python-Bugs-1280061]
in my case (ru_RU)

>>>from sys import platform
>>>from locale import setlocale, LC_ALL
>>>locale = 'russian' if platform == 'win32' else ''
>>>setlocale(LC_ALL, locale) #set locale for '%d %b %Y' convertion
>>>import time, datetime
>>>datetime.date(*time.strptime(u'31 мая 2011', '%d %b %Y')[:3]) # like in django/forms/fields.py, Line 362
ValueError: time data u'31 \u043c\u0430\u044f 2011' does not match format '%d %b %Y'

for strings contains non-ascii characters, you need

'string'.encode('utf8')
>>>datetime.date(*time.strptime(u'31 мая 2011'.encode('utf8'), '%d %b %Y')[:3])
datetime.date(2011, 5, 31)

Attachments (2)

patch.diff (1.3 KB) - added by Gene 4 years ago.
patch_for_ticket16123.diff (1.9 KB) - added by wimfeijen 4 years ago.
Patch for ticket 16123

Download all attachments as: .zip

Change History (9)

Changed 4 years ago by Gene

comment:1 Changed 4 years ago by jonash

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

Seems like the right solution to me.

comment:2 Changed 4 years ago by ramiro

  • Needs tests set
  • Patch needs improvement set

Patch need to be generated from the root of the source code tree, needs tests.

comment:3 Changed 4 years ago by wimfeijen

  • Owner changed from nobody to wimfeijen
  • UI/UX unset

Workon on this during DjangoCon Europe 2011 sprints

comment:4 Changed 4 years ago by wimfeijen

Bug is confirmed against trunk.

Suggested patch wirks, but has to be updated to trunk; and tests are needed. I'll try to write those in the upcoming hour.

# in settings.py:
LANGUAGE_CODE = 'ru'

# BEFORE PATCH:
>>> from django.forms.fields import DateField
>>> from locale import setlocale, LC_ALL
>>> setlocale(LC_ALL, 'ru_RU.utf-8')
'ru_RU.utf-8'
>>> DateField().strptime(u'31 мая 2011', '%d %b %Y')
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/django/forms/fields.py", line 376, in strptime
    return datetime.date(*time.strptime(value, format)[:3])
  File "/usr/lib/python2.7/_strptime.py", line 454, in _strptime_time
    return _strptime(data_string, format)[0]
  File "/usr/lib/python2.7/_strptime.py", line 325, in _strptime
    (data_string, format))
ValueError: time data u'31 \u043c\u0430\u044f 2011' does not match format '%d %b %Y'

# AFTER PATCH:
>>> from django.forms.fields import DateField
>>> from locale import setlocale, LC_ALL
>>> setlocale(LC_ALL, 'ru_RU.utf-8')
'ru_RU.utf-8'
>>> DateField().strptime(u'31 мая 2011', '%d %b %Y')
datetime.date(2011, 5, 31)

Changed 4 years ago by wimfeijen

Patch for ticket 16123

comment:5 Changed 4 years ago by wimfeijen

Ok, I included a new patch which fixes the problem.

Before committing, there are three things to look at:

  1. You need to have the Russian language installed to run the test...
  2. The tests import DateField and DateTimeField. Is this the proper way to do it? If so, the imports should be placed on top op the file. If not, the test should be rewritten.
  3. In class BaseTemporalField, method to_python (django/forms/fields.py, line 347) a similar statement is used:
    datetime.datetime(*time.strptime(datetime_str, format[:-3])[:6]+(usecs,))
    

which might need to be converted to

datetime.datetime(*time.strptime(datetime_str.encode('utf-8'), format[:-3])[:6]+(usecs,))

but I cannot write a test case for this, because I do not know where this is being used.

comment:6 Changed 4 years ago by wimfeijen

  • Owner wimfeijen deleted

comment:7 Changed 2 years ago by Claude Paroz <claude@…>

  • Owner set to Claude Paroz <claude@…>
  • Resolution set to fixed
  • Status changed from new to closed

In 4b3f7110ae1e9ef7862c67a38431780080f0062c:

Fixed #16123 -- Ensured strptime receive proper string type

strptime generates an UnicodeEncodeError when using a non-ascii
unicode string on Python 2.

Note: See TracTickets for help on using tickets.
Back to Top