Context Navigation

← Previous Ticket
Next Ticket →

#28075 closed Bug (fixed)

USE_THOUSAND_SEPARATOR = True, causes foreignkey id to become localized

Reported by:	George Tantiras	Owned by:	Jon Dufresne
Component:	Forms	Version:	1.11
Severity:	Release blocker	Keywords:
Cc:	Jon Dufresne	Triage Stage:	Accepted
Has patch:	yes	Needs documentation:	no
Needs tests:	no	Patch needs improvement:	no
Easy pickings:	no	UI/UX:	no

Description (last modified by George Tantiras)

After upgrading to Django 1.11, when a foreignkey points to an id with more than 3 digits, it gets localized.

In django admin, when I visit an already existing instance and hit "Save and Continue Editing", I receive the following error:

Exception raised while rendering {% include %} for template 'admin/change_form.html'. Empty string rendered instead.
Traceback (most recent call last):
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/loader_tags.py", line 216, in render
    return template.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 209, in render
    return self._render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 199, in _render
    return self.nodelist.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 990, in render
    bit = node.render_annotated(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 957, in render_annotated
    return self.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/defaulttags.py", line 216, in render
    nodelist.append(node.render_annotated(context))
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 957, in render_annotated
    return self.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/defaulttags.py", line 529, in render
    return self.nodelist.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 990, in render
    bit = node.render_annotated(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 957, in render_annotated
    return self.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/defaulttags.py", line 216, in render
    nodelist.append(node.render_annotated(context))
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 957, in render_annotated
    return self.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/defaulttags.py", line 411, in render
    return strip_spaces_between_tags(self.nodelist.render(context).strip())
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 990, in render
    bit = node.render_annotated(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 957, in render_annotated
    return self.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/defaulttags.py", line 322, in render
    return nodelist.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 990, in render
    bit = node.render_annotated(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 957, in render_annotated
    return self.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 1046, in render
    return render_value_in_context(output, context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 1024, in render_value_in_context
    value = force_text(value)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/utils/encoding.py", line 76, in force_text
    s = six.text_type(s)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/utils/html.py", line 385, in <lambda>
    klass.__str__ = lambda self: mark_safe(klass_str(self))
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/forms/boundfield.py", line 41, in __str__
    return self.as_widget()
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/forms/boundfield.py", line 120, in as_widget
    **kwargs
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/forms/widgets.py", line 220, in render
    context = self.get_context(name, value, attrs)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/contrib/admin/widgets.py", line 281, in get_context
    'rendered_widget': self.widget.render(name, value, attrs),
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/dal/widgets.py", line 150, in render
    widget = super(WidgetMixin, self).render(name, value, attrs)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/forms/widgets.py", line 220, in render
    context = self.get_context(name, value, attrs)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/forms/widgets.py", line 665, in get_context
    context = super(Select, self).get_context(name, value, attrs)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/forms/widgets.py", line 627, in get_context
    context['widget']['optgroups'] = self.optgroups(name, context['widget']['value'], attrs)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/dal/widgets.py", line 140, in optgroups
    self.filter_choices_to_render(selected_choices)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/dal/widgets.py", line 183, in filter_choices_to_render
    pk__in=[c for c in selected_choices if c]
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/query.py", line 781, in filter
    return self._filter_or_exclude(False, *args, **kwargs)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/query.py", line 799, in _filter_or_exclude
    clone.query.add_q(Q(*args, **kwargs))
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/sql/query.py", line 1260, in add_q
    clause, _ = self._add_q(q_object, self.used_aliases)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/sql/query.py", line 1286, in _add_q
    allow_joins=allow_joins, split_subq=split_subq,
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/sql/query.py", line 1220, in build_filter
    condition = self.build_lookup(lookups, col, value)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/sql/query.py", line 1114, in build_lookup
    return final_lookup(lhs, rhs)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/lookups.py", line 24, in __init__
    self.rhs = self.get_prep_lookup()
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/lookups.py", line 219, in get_prep_lookup
    rhs_value = self.lhs.output_field.get_prep_value(rhs_value)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/fields/__init__.py", line 962, in get_prep_value
    return int(value)
ValueError: invalid literal for int() with base 10: '1.211'

Changing the setting:

USE_THOUSAND_SEPARATOR = False

renders everything back to normal.

Change History (18)

comment:1 by George Tantiras, 8 years ago

Description:	modified (diff)

comment:2 by George Tantiras, 8 years ago

Description:	modified (diff)

comment:3 by Tim Graham, 8 years ago

In the traceback, I see you're using django-autocomplete-light (dal). Can you reproduce this issue without that package to rule out an issue there? Could you provide a sample project to more easily reproduce the issue. I tried to briefly reproduce it using but putting a foreign key in ModelAdmin.raw_id_fields but couldn't reproduce. Ideally, you could also bisect to find the commit where the behavior changed.

comment:4 by Tim Graham, 8 years ago

Resolution:	→ needsinfo
Status:	new → closed

comment:5 by George Tantiras, 8 years ago

To reproduce the issue, I followed the below steps:

In a fresh django-1.11 installation:

Create the app separ

separ/models.py:

from django.db import models


class Parent(models.Model):
    a_field = models.BooleanField()


class Child(models.Model):
    parent_key = models.ForeignKey(Parent)

separ/admin.py:

from django.contrib import admin
from separ.models import Parent, Child


class ParentAdmin(admin.ModelAdmin):
    pass

admin.site.register(Parent, ParentAdmin)


class ChildAdmin(admin.ModelAdmin):
    pass

admin.site.register(Child, ChildAdmin)

in the settings.py file add the separ module in the INSTALLED_APPS and the line:
```
    USE_THOUSAND_SEPARATOR = True
```

Load the Parent model with data:

>>> from separ.models import Parent, Child
>>> for i in range(1,1500): 
...        a = Parent(a_field = True)
...        a.save()
>>> a = Parent.objects.get(id=1235)
>>> b = Child(parent_key=a)
>>> b.save()

Visit the relevant url and hit 'Save and Add Another":
```
http://127.0.0.1:8000/admin/separ/child/1/change/
```

An error message appears: Select a valid choice. That choice is not one of the available choices.

Trying to debug the error I have found that:

/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/contrib/admin/options.py(1438)_changeform_view()
   1437         ModelForm = self.get_form(request, obj)
-> 1438         if request.method == 'POST':
   1439             form = ModelForm(request.POST, request.FILES, instance=obj)

ipdb> request.POST
<QueryDict: {'parent_key': ['1.235'], ... }

The result of the request.POST command contains the id of the related model localised: ['1.235'].

I have not yet performed a bisect because I have not done it before, but I will give it a try.

Version 0, edited 8 years ago by George Tantiras (next)

comment:6 by George Tantiras, 8 years ago

Summary:	USE_THOUSANDS_SEPARATOR = True, causes foreignkey id to become localized → USE_THOUSAND_SEPARATOR = True, causes foreignkey id to become localized

comment:7 by George Tantiras, 8 years ago

Description:	modified (diff)

comment:8 by George Tantiras, 8 years ago

Resolution:	needsinfo
Status:	closed → new

comment:9 by Tim Graham, 8 years ago

Which locale are you using?

comment:10 by George Tantiras, 8 years ago

In the example, am using the settings.py as it comes with the default django installation.
However, if it might make sense, I have the following files in my project tree, for the locales en, el:

<project name>formats/en/formats.py, <project name>formats/el/formats.py

DATE_FORMAT = 'd-m-Y'
TIME_FORMAT = 'H:i'
DATETIME_FORMAT = 'd-m-Y H:i'
YEAR_MONTH_FORMAT = 'F Y'
MONTH_DAY_FORMAT = 'F j'
SHORT_DATE_FORMAT = 'm/d/Y'
SHORT_DATETIME_FORMAT = 'm/d/Y P'
FIRST_DAY_OF_WEEK = 1

# BUT here use the Python strftime format syntax,
# LIST: http://docs.python.org/library/datetime.html#strftime-strptime-behavior

DATE_INPUT_FORMATS = (
    '%d-%m-%Y',     # '21-03-2014'
)
TIME_INPUT_FORMATS = (
    '%H:%M:%S',     # '17:59:59'
    '%H:%M',        # '17:59'
)
DATETIME_INPUT_FORMATS = (
    '%d-%m-%Y %H:%M',     # '21-03-2014 17:59'
)

DECIMAL_SEPARATOR = ','
THOUSAND_SEPARATOR = '.'
NUMBER_GROUPING = 3