Document why you cannot iterate defaultdict in templates

[jacob.ninja.dev@…]

I am unable to iterate a defaultdict using a for loop in a template.

Code to reproduce issue:

dictionary = defaultdict(list)
dictionary['foo'].append('bar')

{% for key, value in dictionary.items %}
  {# Never iterates #}
{% endfor %}

[Matt McClanahan]

Right, because dictionary key access takes precedence in template variable lookups, so {% ... dictionary.items %} adds an empty list named 'items' to the dictionary. Not sure there's a good workaround here.

[Aymeric Augustin]

Indeed, it boils down to the fact that the template language uses the same syntax for dictionary and attribute lookups. The resolution order is documented here: ​https://docs.djangoproject.com/en/1.3/topics/templates/#variables. It's probably a bad idea to change it.

We could add a note there to warn about objects that accept a dictionary lookup with any name, suggesting to convert them to dict before passing them to the view, in your example:

context['dictionary'] = dict(dictionary)

[Nick Meharry]

How much would break if we were to do the defaultdict -> dict conversion automatically?

[Nick Meharry]

Convert defaultdicts to dicts before template lookups.

[Nick Meharry]

Add example of current template behavior w.r.t. defaultdicts.

[Nick Meharry]

After some consideration, perhaps performing the conversion is not such a good idea. I've added an example to the docs that explains why the current behavior happens.

[Tomek Paczkowski]

Patch with docs explains issue nicely.

[anonymous]

Patch looks great, however just to be super clear, I think it should include a bit like, "So while this looks like it should do defaultdict.iteritems() in fact it's doing defaultdictiteritems which returns the default value".

[Chris Beaven]

Hrm, I wonder if the _resolve_lookup method could be made smarter with something like the following for the dictionary lookup section:

try:
    if bit not in current:
        raise KeyError
except TypeError:
    pass
current = current[bit]

[Aymeric Augustin]

Fixed #16335 -- Clarified an unintuitive behavior.

The DTL will perform dict lookup before method lookup, which yields
an unexpected result for defaultdicts.

Changeset: d171b3cc0b32374fd5891254b04693e9ec8ed946

[Aymeric Augustin]

Fixed #16335 -- Clarified an unintuitive behavior.

The DTL will perform dict lookup before method lookup, which yields
an unexpected result for defaultdicts.

Changeset: d171b3cc0b32374fd5891254b04693e9ec8ed946

[Aymeric Augustin]

Fixed #16335 -- Clarified an unintuitive behavior.

The DTL will perform dict lookup before method lookup, which yields
an unexpected result for defaultdicts.

Changeset: d171b3cc0b32374fd5891254b04693e9ec8ed946

[anonymous]

Having this case:

my_dict = collections.defaultdict(lambda: collections.defaultdict(list))    
dictionary['foo']['foo1'].append(obj)

and doing

return Response({'my_dict ': dict(my_dict)}, template_name='_internal_template.html')

I'm able to loop the first for only

{% for groups in my_dict.itervalues  %}
    groups = {{ groups }} <br><br> //It prints: groups = defaultdict(<type 'list'>, {1: [<Obj: Obj 1 by me>]})  
    {% for cols in groups.itervalues  %}
      cols = {{ cols }} <br><br>
      {% for maps in cols.itervalues  %}

[krommail@…]

Django 1.6, Pyhton 3.3.3. Even in the simpliest case:
{% for key, values in dict.items %}

{{ key }}

{% endfor %}
there is nothing in "key".

While
{% for key in dict %}

{{ key }}

{% endfor %}

works well.
Also preliminary conversion of a defaultdict to dict helps.

[ericpulvino@…]

I am also seeing this for Django 1.6.1 python 2.7.6. Conversion to standard dict does not help in my case. Otherwise same symptoms as the post from krommail.

[ericpulvino@…]

After upgrading to 1.6.5 still with python 2.7.6 the workaround (converting to standard dictionary prior to handing dict to the template) functions.

As a Django newbie I have sunk 5 hours on this silly issue. I need a break today.

[Tim Graham]

The documentation that was added as part of the ticket [d171b3cc0b32374fd5891254b04693e9ec8ed946] explains why it doesn't work.

[diego.gaustein@…]

I am actually getting a MemoryError (which sometimes end in segfault) when trying to iterate the values in a defaultdict

In views.py:

totals = defaultdict(Decimal)
add_stuff_to_it(totals)

In the template:

    {% for t in totals.values %}
    {{ t }}
    {% endfor %}

Results in

$ python manage.py runserver 0.0.0.0:8000 --settings=xxx.settings_debug
Validating models...

0 errors found
July 30, 2014 - 09:57:05
Django version 1.6.5, using settings 'xxx.settings_debug'
Starting development server at http://0.0.0.0:8000/
Quit the server with CONTROL-C.
Traceback (most recent call last):
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/contrib/staticfiles/handlers.py", line 67, in __call__
    return self.application(environ, start_response)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 206, in __call__
    response = self.get_response(request)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 194, in get_response
    response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 229, in handle_uncaught_exception
    return debug.technical_500_response(request, *exc_info)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/views/debug.py", line 69, in technical_500_response
    html = reporter.get_traceback_html()
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/views/debug.py", line 324, in get_traceback_html
    return t.render(c)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/template/base.py", line 140, in render
    return self._render(context)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/template/base.py", line 134, in _render
    return self.nodelist.render(context)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/template/base.py", line 840, in render
    bit = self.render_node(node, context)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/template/debug.py", line 78, in render_node
    return node.render(context)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/template/defaulttags.py", line 305, in render
    return nodelist.render(context)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/template/base.py", line 840, in render
    bit = self.render_node(node, context)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/template/debug.py", line 78, in render_node
    return node.render(context)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/template/defaulttags.py", line 36, in render
    output = self.nodelist.render(context)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/template/base.py", line 840, in render
    bit = self.render_node(node, context)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/template/debug.py", line 78, in render_node
    return node.render(context)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/template/defaulttags.py", line 212, in render
    return nodelist.render(context)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/template/base.py", line 844, in render
    return mark_safe(''.join(bits))
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/utils/safestring.py", line 116, in mark_safe
    return SafeText(s)
MemoryError
[30/Jul/2014 09:57:16] "GET /statistics/order_book_and_wip/ HTTP/1.1" 500 59

Strangely, iterating over the keys seems to work fine.
Setting the default_factory to None as detailed ​here makes it work as expected. It would seem like a cheap and sensible workaround to apply that to defaultdicts. Or otherwise at least raise an exception so users can know what's going on.