Opened 6 years ago

Closed 4 years ago

#16123 closed Bug (fixed)

DateTimeField, DateField fails covert unicode date string with non-ascii characters

Reported by: Gene Owned by: Claude Paroz <claude@…>
Component: Forms Version: 1.3
Severity: Normal Keywords: unicode, date, forms
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

it's old python bug - [python-Bugs-1280061]
in my case (ru_RU)

>>>from sys import platform
>>>from locale import setlocale, LC_ALL
>>>locale = 'russian' if platform == 'win32' else ''
>>>setlocale(LC_ALL, locale) #set locale for '%d %b %Y' convertion
>>>import time, datetime
>>>datetime.date(*time.strptime(u'31 мая 2011', '%d %b %Y')[:3]) # like in django/forms/fields.py, Line 362
ValueError: time data u'31 \u043c\u0430\u044f 2011' does not match format '%d %b %Y'

for strings contains non-ascii characters, you need

'string'.encode('utf8')
>>>datetime.date(*time.strptime(u'31 мая 2011'.encode('utf8'), '%d %b %Y')[:3])
datetime.date(2011, 5, 31)

Attachments (2)

patch.diff (1.3 KB) - added by Gene 6 years ago.
patch_for_ticket16123.diff (1.9 KB) - added by Wim Feijen 5 years ago.
Patch for ticket 16123

Download all attachments as: .zip

Change History (9)

Changed 6 years ago by Gene

Attachment: patch.diff added

comment:1 Changed 6 years ago by Jonas H.

Triage Stage: UnreviewedAccepted

Seems like the right solution to me.

comment:2 Changed 6 years ago by Ramiro Morales

Needs tests: set
Patch needs improvement: set

Patch need to be generated from the root of the source code tree, needs tests.

comment:3 Changed 5 years ago by Wim Feijen

Owner: changed from nobody to Wim Feijen
UI/UX: unset

Workon on this during DjangoCon Europe 2011 sprints

comment:4 Changed 5 years ago by Wim Feijen

Bug is confirmed against trunk.

Suggested patch wirks, but has to be updated to trunk; and tests are needed. I'll try to write those in the upcoming hour.

# in settings.py:
LANGUAGE_CODE = 'ru'

# BEFORE PATCH:
>>> from django.forms.fields import DateField
>>> from locale import setlocale, LC_ALL
>>> setlocale(LC_ALL, 'ru_RU.utf-8')
'ru_RU.utf-8'
>>> DateField().strptime(u'31 мая 2011', '%d %b %Y')
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/django/forms/fields.py", line 376, in strptime
    return datetime.date(*time.strptime(value, format)[:3])
  File "/usr/lib/python2.7/_strptime.py", line 454, in _strptime_time
    return _strptime(data_string, format)[0]
  File "/usr/lib/python2.7/_strptime.py", line 325, in _strptime
    (data_string, format))
ValueError: time data u'31 \u043c\u0430\u044f 2011' does not match format '%d %b %Y'

# AFTER PATCH:
>>> from django.forms.fields import DateField
>>> from locale import setlocale, LC_ALL
>>> setlocale(LC_ALL, 'ru_RU.utf-8')
'ru_RU.utf-8'
>>> DateField().strptime(u'31 мая 2011', '%d %b %Y')
datetime.date(2011, 5, 31)

Changed 5 years ago by Wim Feijen

Attachment: patch_for_ticket16123.diff added

Patch for ticket 16123

comment:5 Changed 5 years ago by Wim Feijen

Ok, I included a new patch which fixes the problem.

Before committing, there are three things to look at:

  1. You need to have the Russian language installed to run the test...
  2. The tests import DateField and DateTimeField. Is this the proper way to do it? If so, the imports should be placed on top op the file. If not, the test should be rewritten.
  3. In class BaseTemporalField, method to_python (django/forms/fields.py, line 347) a similar statement is used:
    datetime.datetime(*time.strptime(datetime_str, format[:-3])[:6]+(usecs,))
    

which might need to be converted to

datetime.datetime(*time.strptime(datetime_str.encode('utf-8'), format[:-3])[:6]+(usecs,))

but I cannot write a test case for this, because I do not know where this is being used.

comment:6 Changed 5 years ago by Wim Feijen

Owner: Wim Feijen deleted

comment:7 Changed 4 years ago by Claude Paroz <claude@…>

Owner: set to Claude Paroz <claude@…>
Resolution: fixed
Status: newclosed

In 4b3f7110ae1e9ef7862c67a38431780080f0062c:

Fixed #16123 -- Ensured strptime receive proper string type

strptime generates an UnicodeEncodeError when using a non-ascii
unicode string on Python 2.

Note: See TracTickets for help on using tickets.
Back to Top