USE_THOUSAND_SEPARATOR = True, causes foreignkey id to become localized

[George Tantiras]

After upgrading to Django 1.11, when a foreignkey points to an id with more than 3 digits, it gets localized.

In django admin, when I visit an already existing instance and hit "Save and Continue Editing", I receive the following error:

Exception raised while rendering {% include %} for template 'admin/change_form.html'. Empty string rendered instead.
Traceback (most recent call last):
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/loader_tags.py", line 216, in render
    return template.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 209, in render
    return self._render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 199, in _render
    return self.nodelist.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 990, in render
    bit = node.render_annotated(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 957, in render_annotated
    return self.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/defaulttags.py", line 216, in render
    nodelist.append(node.render_annotated(context))
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 957, in render_annotated
    return self.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/defaulttags.py", line 529, in render
    return self.nodelist.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 990, in render
    bit = node.render_annotated(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 957, in render_annotated
    return self.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/defaulttags.py", line 216, in render
    nodelist.append(node.render_annotated(context))
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 957, in render_annotated
    return self.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/defaulttags.py", line 411, in render
    return strip_spaces_between_tags(self.nodelist.render(context).strip())
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 990, in render
    bit = node.render_annotated(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 957, in render_annotated
    return self.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/defaulttags.py", line 322, in render
    return nodelist.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 990, in render
    bit = node.render_annotated(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 957, in render_annotated
    return self.render(context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 1046, in render
    return render_value_in_context(output, context)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/template/base.py", line 1024, in render_value_in_context
    value = force_text(value)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/utils/encoding.py", line 76, in force_text
    s = six.text_type(s)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/utils/html.py", line 385, in <lambda>
    klass.__str__ = lambda self: mark_safe(klass_str(self))
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/forms/boundfield.py", line 41, in __str__
    return self.as_widget()
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/forms/boundfield.py", line 120, in as_widget
    **kwargs
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/forms/widgets.py", line 220, in render
    context = self.get_context(name, value, attrs)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/contrib/admin/widgets.py", line 281, in get_context
    'rendered_widget': self.widget.render(name, value, attrs),
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/dal/widgets.py", line 150, in render
    widget = super(WidgetMixin, self).render(name, value, attrs)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/forms/widgets.py", line 220, in render
    context = self.get_context(name, value, attrs)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/forms/widgets.py", line 665, in get_context
    context = super(Select, self).get_context(name, value, attrs)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/forms/widgets.py", line 627, in get_context
    context['widget']['optgroups'] = self.optgroups(name, context['widget']['value'], attrs)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/dal/widgets.py", line 140, in optgroups
    self.filter_choices_to_render(selected_choices)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/dal/widgets.py", line 183, in filter_choices_to_render
    pk__in=[c for c in selected_choices if c]
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/query.py", line 781, in filter
    return self._filter_or_exclude(False, *args, **kwargs)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/query.py", line 799, in _filter_or_exclude
    clone.query.add_q(Q(*args, **kwargs))
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/sql/query.py", line 1260, in add_q
    clause, _ = self._add_q(q_object, self.used_aliases)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/sql/query.py", line 1286, in _add_q
    allow_joins=allow_joins, split_subq=split_subq,
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/sql/query.py", line 1220, in build_filter
    condition = self.build_lookup(lookups, col, value)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/sql/query.py", line 1114, in build_lookup
    return final_lookup(lhs, rhs)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/lookups.py", line 24, in __init__
    self.rhs = self.get_prep_lookup()
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/lookups.py", line 219, in get_prep_lookup
    rhs_value = self.lhs.output_field.get_prep_value(rhs_value)
  File "/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/db/models/fields/__init__.py", line 962, in get_prep_value
    return int(value)
ValueError: invalid literal for int() with base 10: '1.211'

Changing the setting:

USE_THOUSAND_SEPARATOR = False

renders everything back to normal.

[Tim Graham]

In the traceback, I see you're using django-autocomplete-light (dal). Can you reproduce this issue without that package to rule out an issue there? Could you provide a sample project to more easily reproduce the issue. I tried to briefly reproduce it using but putting a foreign key in ModelAdmin.raw_id_fields but couldn't reproduce. Ideally, you could also ​bisect to find the commit where the behavior changed.

[George Tantiras]

To reproduce the issue, I followed the below steps:

In a fresh Django-1.11 installation:

• Create the app separ
• separ/models.py:

  from django.db import models
  
  
  class Parent(models.Model):
      a_field = models.BooleanField()
  
  
  class Child(models.Model):
      parent_key = models.ForeignKey(Parent)
  

• separ/admin.py:

  from django.contrib import admin
  from separ.models import Parent, Child
  
  
  class ParentAdmin(admin.ModelAdmin):
      pass
  
  admin.site.register(Parent, ParentAdmin)
  
  
  class ChildAdmin(admin.ModelAdmin):
      pass
  
  admin.site.register(Child, ChildAdmin)
  

• in the settings.py file add the separ module in the INSTALLED_APPS and the line:

      USE_THOUSAND_SEPARATOR = True
  

• Load the Parent model with data:

  >>> from separ.models import Parent, Child
  >>> for i in range(1,1500): 
  ...        a = Parent(a_field = True)
  ...        a.save()
  >>> a = Parent.objects.get(id=1235)
  >>> b = Child(parent_key=a)
  >>> b.save()
  

• Visit the relevant url and hit 'Save and Add Another":

  http://127.0.0.1:8000/admin/separ/child/1/change/
  

An error message appears: Select a valid choice. That choice is not one of the available choices.

Trying to debug the error I have found that:

/home/flyer/.virtualenvs/erad/lib/python3.4/site-packages/django/contrib/admin/options.py(1438)_changeform_view()
   1437         ModelForm = self.get_form(request, obj)
-> 1438         if request.method == 'POST':
   1439             form = ModelForm(request.POST, request.FILES, instance=obj)

ipdb> request.POST
<QueryDict: {'parent_key': ['1.235'], ... }

The result of the request.POST command contains the id of the related model localised: ['1.235'].

I have not yet performed a bisect because I have not done it before, but I will give it a try.