Prefetch can execute an UPDATE statement if a certain to_attr is given

[kirillnedelev]

Following a django tutorial I've created polls application.
The only thing that was changed was a ForeignKey between Question and Choice models, null=True was added.

from django.db import models

class Question(models.Model):
    question_text = models.CharField(max_length=200)
    pub_date = models.DateTimeField('date published')


class Choice(models.Model):
    question = models.ForeignKey(Question, blank=True,null=True)
    choice_text = models.CharField(max_length=200)
    votes = models.IntegerField(default=0)

From the shell I've created one Question and one Choice:

from polls.models import Question, Choice
from django.utils import timezone

q = Question(question_text="Question", pub_date=timezone.now())
q.save()

q.choice_set.create(choice_text='Choice', votes=0)

After that I've executed:

from django.db.models import Prefetch

questions = (Question.objects
    .all()
    .prefetch_related(
        Prefetch('choice_set',
            queryset=(Choice.objects.filter(votes=5)),
            to_attr='choice_set'
        )
    )
)

print questions

And here are the queries that were executed:

(0.003) QUERY = u'SELECT "polls_question"."id", "polls_question"."question_text", "polls_question"."pub_date" FROM "polls_question" LIMIT 21' - PARAMS = (); args=()
(0.000) QUERY = u'SELECT "polls_choice"."id", "polls_choice"."question_id", "polls_choice"."choice_text", "polls_choice"."votes" FROM "polls_choice" WHERE ("polls_choice"."votes" = %s AND "polls_choice"."question_id" IN (%s))' - PARAMS = (5, 1); args=(5, 1)
(0.000) QUERY = u'BEGIN' - PARAMS = (); args=None
(0.001) QUERY = u'UPDATE "polls_choice" SET "question_id" = NULL WHERE "polls_choice"."question_id" = %s' - PARAMS = (1,); args=(1,)

UPDATE query sets all question_id to NULL, which means that my question model doesn't have any choices now

q = Question(pk=1)
q.choice_set.all()
# []

I've traced the method where UPDATE statement was executed
django/db/models/query.py

def prefetch_one_level(instances, prefetcher, lookup, level):
...
    for obj in instances:
        instance_attr_val = instance_attr(obj)
        vals = rel_obj_cache.get(instance_attr_val, [])

        if single:
            val = vals[0] if vals else None
            to_attr = to_attr if as_attr else cache_name
            setattr(obj, to_attr, val)
        else:
            if as_attr:
->              setattr(obj, to_attr, vals)
            else:
                manager = getattr(obj, to_attr)
                if leaf and lookup.queryset is not None:
                    qs = manager._apply_rel_filters(lookup.queryset)
                else:
                    qs = manager.get_queryset()
                qs._result_cache = vals
                # We don't want the individual qs doing prefetch_related now,
                # since we have merged this into the current work.
                qs._prefetch_done = True
                obj._prefetched_objects_cache[cache_name] = qs
...

If I remove to_attr parameter from Prefetch, it works fine

questions = (Question.objects
    .all()
    .prefetch_related(
        Prefetch('choice_set',
            queryset=(Choice.objects.filter(votes=5))
            #,to_attr='choice_set'
        )
    )
)

But still it's risky behavior.

[Tim Graham]

This is fixed in master (Django 2.0) by ed251246cc6a22561217f38f7cf96598b22ff0fe. If someone provided a patch for older versions of Django to raise an error for a clashing to_attr to fix the data loss possibility, that would be accepted.

[Simon Charette]

Given no one as provided a patch for 1.11 I believe we can close this at this point.