sql_queries from debug template context processor always empty

[Markus Holtermann]

The sql_queries template context variable should propagated with all the queries from connection.queries. However, this list seems to be always empty:

# django.core.context_processors
def debug(request):
    "Returns context variables helpful for debugging."
    context_extras = {}
    import ipdb; ipdb.set_trace()
    if settings.DEBUG and request.META.get('REMOTE_ADDR') in settings.INTERNAL_IPS:
        context_extras['debug'] = True
        from django.db import connection
        context_extras['sql_queries'] = connection.queries
    return context_extras

[Markus Holtermann]

Example code

[Aymeric Augustin]

This is interesting.

Here's the relevant template from the sample project. mymodels is a queryset and it isn't evaluated yet. sql_queries is set to connection.queries in the debug context processor.

{% for mymodel in mymodels %}
  <li>{{ mymodel.foo }}</li>
{% endfor %}
<pre>
{{ sql_queries }}
{{ sql_queries|length }}
</pre>

In Django 1.7, connection.queries is a list that gets updated with each new database query. If you print the list in the context processor, it's empty. But by the time you reach {{ sql_queries }} in the template, {% for mymodel in mymodels %} has triggered a query, and {{ sql_queries }} contains that query.

In Django pre-1.8, connection.queries is a snapshot of the ring buffer that prevents unlimited memory consumption in long running processes. Once the snapshot is made, it isn't updated with new database queries. As a consequence, it's still empty when reaching {{ sql_queries }} in the template.

I'm not sure what to do. Since it's common to have database queries triggered by templates in Django projects, this unintentional change of behavior will be considered a regression by most users. But having {{ sql_queries }} return different values depending on where you put it in the template isn't a good API either.

[Bas Peschier]

Made sql_queries lazy and added tests for the debug context processor: ​https://github.com/bpeschier/django/compare/ticket_23364

[Aymeric Augustin]

Looks good! I'll take it from there.

[Aymeric Augustin <aymeric.augustin@…>]

In eacf244506d8b7e6dd6483834ea122fec864da85:

Converted sql_queries into a lazily evaluated list.

Fixed #23364. Thanks Markush2010 for the report.