Opened 6 months ago

Closed 6 months ago

#23364 closed Bug (fixed)

sql_queries from debug template context processor always empty

Reported by: Markush2010 Owned by: aaugustin
Component: HTTP handling Version: master
Severity: Normal Keywords:
Cc: Markush2010 Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

The sql_queries template context variable should propagated with all the queries from connection.queries. However, this list seems to be always empty:

# django.core.context_processors
def debug(request):
    "Returns context variables helpful for debugging."
    context_extras = {}
    import ipdb; ipdb.set_trace()
    if settings.DEBUG and request.META.get('REMOTE_ADDR') in settings.INTERNAL_IPS:
        context_extras['debug'] = True
        from django.db import connection
        context_extras['sql_queries'] = connection.queries
    return context_extras

Attachments (1)

djtest2.tar.gz (60.0 KB) - added by Markush2010 6 months ago.
Example code

Download all attachments as: .zip

Change History (6)

Changed 6 months ago by Markush2010

Example code

comment:1 Changed 6 months ago by aaugustin

  • Component changed from Uncategorized to HTTP handling
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

This is interesting.

Here's the relevant template from the sample project. mymodels is a queryset and it isn't evaluated yet. sql_queries is set to connection.queries in the debug context processor.

{% for mymodel in mymodels %}
  <li>{{ mymodel.foo }}</li>
{% endfor %}
<pre>
{{ sql_queries }}
{{ sql_queries|length }}
</pre>

In Django 1.7, connection.queries is a list that gets updated with each new database query. If you print the list in the context processor, it's empty. But by the time you reach {{ sql_queries }} in the template, {% for mymodel in mymodels %} has triggered a query, and {{ sql_queries }} contains that query.

In Django pre-1.8, connection.queries is a snapshot of the ring buffer that prevents unlimited memory consumption in long running processes. Once the snapshot is made, it isn't updated with new database queries. As a consequence, it's still empty when reaching {{ sql_queries }} in the template.

I'm not sure what to do. Since it's common to have database queries triggered by templates in Django projects, this unintentional change of behavior will be considered a regression by most users. But having {{ sql_queries }} return different values depending on where you put it in the template isn't a good API either.

comment:2 Changed 6 months ago by Markush2010

  • Cc Markush2010 added

comment:3 Changed 6 months ago by bpeschier

  • Has patch set

Made sql_queries lazy and added tests for the debug context processor: https://github.com/bpeschier/django/compare/ticket_23364

comment:4 Changed 6 months ago by aaugustin

  • Owner changed from nobody to aaugustin
  • Status changed from new to assigned

Looks good! I'll take it from there.

comment:5 Changed 6 months ago by Aymeric Augustin <aymeric.augustin@…>

  • Resolution set to fixed
  • Status changed from assigned to closed

In eacf244506d8b7e6dd6483834ea122fec864da85:

Converted sql_queries into a lazily evaluated list.

Fixed #23364. Thanks Markush2010 for the report.

Note: See TracTickets for help on using tickets.
Back to Top