Opened 10 years ago

Closed 10 years ago

Last modified 10 years ago

#23112 closed Bug (fixed)

Field.get_choices tries to index an iterable: itertools.tee' object has no attribute '__getitem__'

Reported by: bernie_sumption Owned by: Areski Belaid
Component: Database layer (models, ORM) Version: 1.7-rc-2
Severity: Release blocker Keywords:
Cc: Florian Apolloner, areski@…, pdewacht@…, lemuelf@…, cmawebsite@…, Jcuotpc@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Issue #23098 was fixed with this commit: https://github.com/django/django/commit/2f73b527dda6683868fac2791f7f07ccb01ea0d9

Which adds the following line:

named_groups = self.choices and isinstance(self.choices[0][1], (list, tuple))

According to the docs - https://docs.djangoproject.com/en/dev/ref/models/fields/#choices - Field.choices can be any iterable of choices. The above line introduces the requirement that the choices object to be subscriptable, and breaks both my own code and a 3rd party library I'm using (django_countries).

Change History (23)

comment:1 by Mathieu Agopian, 10 years ago

Also, we can have a list of choices made of both named groups and "normal" choices, so we might no "see" that the blank choice is defined in the list of choices (it would be odd to not have it at the beginning of the list, but i don't think it's required, at least from the previous code and the docs).

comment:2 by Tim Graham, 10 years ago

Triage Stage: UnreviewedAccepted
Version: master1.7-rc-2

in reply to:  1 comment:3 by Florian Apolloner, 10 years ago

Replying to magopian:

Also, we can have a list of choices made of both named groups and "normal" choices, so we might no "see" that the blank choice is defined in the list of choices (it would be odd to not have it at the beginning of the list, but i don't think it's required, at least from the previous code and the docs).

That didn't work either previously, so we are somewhat safe in that regard… The original iterable argument seems valid, we can just call list on it I guess.

comment:4 by Florian Apolloner, 10 years ago

Cc: Florian Apolloner added

comment:5 by Areski Belaid, 10 years ago

Cc: areski@… added

I could reproduce the issue using an iterator, here a PR that fix the issue at my end
https://github.com/django/django/pull/2990

Thanks, I used list as @apollo13 mentioned.

Last edited 10 years ago by Areski Belaid (previous) (diff)

comment:6 by Areski Belaid, 10 years ago

Has patch: set

The PR now contains tests: https://github.com/django/django/pull/2990
I noticed that with Iterators, we cannot use _get_FIELD_display properly, now this problem might not be blocking.

comment:7 by Zbigniew Siciarz, 10 years ago

I got bitten by this bug too (talked about it yesterday with Baptiste). Here's a traceback from my project which used to work on RC1, breaks on RC2.

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-packages/django/core/management/__init__.py", line 354, in execute
    django.setup()
  File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-packages/django/__init__.py", line 21, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-packages/django/apps/registry.py", line 115, in populate
    app_config.ready()
  File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-packages/django/contrib/admin/apps.py", line 22, in ready
    self.module.autodiscover()
  File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-packages/django/contrib/admin/__init__.py", line 23, in autodiscover
    autodiscover_modules('admin', register_to=site)
  File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-packages/django/utils/module_loading.py", line 74, in autodiscover_modules
    import_module('%s.%s' % (app_config.name, module_to_search))
  File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1584, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1565, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 584, in _check_name_wrapper
  File "<frozen importlib._bootstrap>", line 1022, in load_module
  File "<frozen importlib._bootstrap>", line 1003, in load_module
  File "<frozen importlib._bootstrap>", line 560, in module_for_loader_wrapper
  File "<frozen importlib._bootstrap>", line 868, in _load_module
  File "<frozen importlib._bootstrap>", line 313, in _call_with_frames_removed
  File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-packages/allauth/socialaccount/admin.py", line 12, in <module>
    class SocialAppForm(forms.ModelForm):
  File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-packages/django/forms/models.py", line 284, in __new__
    opts.help_texts, opts.error_messages)
  File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-packages/django/forms/models.py", line 210, in fields_for_model
    formfield = f.formfield(**kwargs)
  File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-packages/django/db/models/fields/__init__.py", line 1063, in formfield
    return super(CharField, self).formfield(**defaults)
  File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-packages/django/db/models/fields/__init__.py", line 822, in formfield
    defaults['choices'] = self.get_choices(include_blank=include_blank)
  File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-packages/django/db/models/fields/__init__.py", line 733, in get_choices
    named_groups = self.choices and isinstance(self.choices[0][1], (list, tuple))
TypeError: 'itertools._tee' object is not subscriptable

comment:8 by Areski Belaid, 10 years ago

Did you try with this patch https://github.com/django/django/pull/2990?

comment:9 by Peter De Wachter, 10 years ago

Cc: pdewacht@… added

comment:10 by Zbigniew Siciarz, 10 years ago

Yes, wrapping self.choices with a list() call fixed the problem.

comment:11 by Areski Belaid, 10 years ago

Update of the PR thanks to the feedback from @Apollo13
https://github.com/django/django/pull/2990

comment:12 by Areski Belaid, 10 years ago

Owner: changed from nobody to Areski Belaid
Status: newassigned

comment:13 by Areski Belaid, 10 years ago

Following @timgraham advice, the commented tests has been removed from the PR

comment:14 by Lemuel Formacil, 10 years ago

Cc: lemuelf@… added

comment:15 by Collin Anderson, 10 years ago

Cc: cmawebsite@… added

comment:16 by Collin Anderson, 10 years ago

As noticed in #23119, the patch doesn't work if the iterable is empty (empty iterable is fine on 1.6)

field = models.CharField(choices=(x for x in []), blank=True, max_length=1)

raises IndexError: list index out of range with the pull request. No problem on 1.6

comment:17 by Collin Anderson, 10 years ago

Summary: Field.get_choices tries to index an iterableField.get_choices tries to index an iterable: itertools.tee' object has no attribute '__getitem__'

comment:18 by Areski Belaid, 10 years ago

Thanks @CollinAnderson check the pull request, I just added a fix for empty iterators and also some regression tests

comment:19 by Julian C, 10 years ago

Cc: Jcuotpc@… added

comment:20 by Julian C, 10 years ago

Cc: Jcuotpc@… added

comment:21 by Collin Anderson, 10 years ago

The pull request works for me. I closed #23119 as duplicate.

comment:22 by Florian Apolloner <florian@…>, 10 years ago

Resolution: fixed
Status: assignedclosed

In 97a38de230371c0b6ad8a86abba8425186c147c7:

Fixed #23112 -- Field.get_choices tries to index an iterable

comment:23 by Florian Apolloner <florian@…>, 10 years ago

In e22ad1c3250a59d046d73bf34ae39c0c40e03c98:

[1.7.x] Fixed #23112 -- Field.get_choices tries to index an iterable

Backport of 97a38de230371c0b6ad8a86abba8425186c147c7 from master.

Note: See TracTickets for help on using tickets.
Back to Top