Can no longer query reverse generic relations in Django 1.6

[khoobks@…]

Please see the code below for a demonstration of the regression.

The Model

from django.db import models
from django.contrib.contenttypes.models import ContentType
from django.contrib.contenttypes import generic

class TaggedItem(models.Model):
    tag = models.SlugField()
    content_type = models.ForeignKey(ContentType)
    object_id = models.PositiveIntegerField()
    content_object = generic.GenericForeignKey('content_type', 'object_id')

    # On Python 3: def __str__(self):
    def __unicode__(self):
        return self.tag

class Bookmark(models.Model):
    url = models.URLField()
    tags = generic.GenericRelation(TaggedItem)

The Setup

from test_app.models import *

b = Bookmark(url='https://www.djangoproject.com/')
b.save()
t1 = TaggedItem(content_object=b, tag='django')
t1.save()
t2 = TaggedItem(content_object=b, tag='python')
t2.save()

Django 1.5

Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import django
>>> from test_app.models import *
>>> django.VERSION
(1, 5, 5, 'final', 0)
>>> TaggedItem.objects.filter(bookmark__url__icontains='django')
[<TaggedItem: django>, <TaggedItem: python>]
>>>

Django 1.6

Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>>
>>> import django
>>> from test_app.models import *
>>> django.VERSION
(1, 6, 0, 'final', 0)
>>> TaggedItem.objects.filter(bookmark__url__icontains='django')
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "C:\workspace\django_generic_relationship\django16\lib\site-packages\django\db\models\manager.py", line 163, in f
ilter
    return self.get_queryset().filter(*args, **kwargs)
  File "C:\workspace\django_generic_relationship\django16\lib\site-packages\django\db\models\query.py", line 590, in fil
ter
    return self._filter_or_exclude(False, *args, **kwargs)
  File "C:\workspace\django_generic_relationship\django16\lib\site-packages\django\db\models\query.py", line 608, in _fi
lter_or_exclude
    clone.query.add_q(Q(*args, **kwargs))
  File "C:\workspace\django_generic_relationship\django16\lib\site-packages\django\db\models\sql\query.py", line 1198, i
n add_q
    clause = self._add_q(where_part, used_aliases)
  File "C:\workspace\django_generic_relationship\django16\lib\site-packages\django\db\models\sql\query.py", line 1232, i
n _add_q
    current_negated=current_negated)
  File "C:\workspace\django_generic_relationship\django16\lib\site-packages\django\db\models\sql\query.py", line 1100, i
n build_filter
    allow_explicit_fk=True)
  File "C:\workspace\django_generic_relationship\django16\lib\site-packages\django\db\models\sql\query.py", line 1351, i
n setup_joins
    names, opts, allow_many, allow_explicit_fk)
  File "C:\workspace\django_generic_relationship\django16\lib\site-packages\django\db\models\sql\query.py", line 1274, i
n names_to_path
    "Choices are: %s" % (name, ", ".join(available)))
FieldError: Cannot resolve keyword 'bookmark' into field. Choices are: content_type, id, object_id, tag
>>>

I would have expected that the same code should work the same way between Django 1.5 and 1.6.

[Anssi Kääriäinen]

I can confirm that the queries work differently. However, the 1.5 version didn't work correctly at all. Using generic_relations test models:

    a = Animal.objects.create(common_name='foo', latin_name='bar')
    Vegetable.objects.create(name='foo', pk=a.pk)
    TaggedItem.objects.create(content_object=a)
    print TaggedItem.objects.filter(vegetable__name__icontains='foo')
    print TaggedItem.objects.filter(vegetable__name__icontains='foo').query

The output is:

[<TaggedItem: >]
SELECT ...
  FROM "generic_relations_taggeditem"
 INNER JOIN "generic_relations_taggeditem" T2 ON ("generic_relations_taggeditem"."id" = T2."id")
 INNER JOIN "generic_relations_vegetable" ON (T2."object_id" = "generic_relations_vegetable"."id")
 INNER JOIN "django_content_type" ON
       ("generic_relations_taggeditem"."content_type_id" = "django_content_type"."id")
 WHERE "generic_relations_vegetable"."name" LIKE %foo% ESCAPE '\' 
 ORDER BY "generic_relations_taggeditem"."tag" ASC, "django_content_type"."name" ASC

Note the lack of any constraint on the content type. For this reason a vegetable is found even if there is no relationship to any vegetable at all.

The query construct used wasn't documented (at least I can't find a reference) and is completely untested. I am confident that having the query produce any results at all is just an accident caused by implementation details.

I think I'll just have to document that such queries do not work any more, and that they never actually produced any correct results. The other option is to make this construct work properly in 1.6, but I believe that will be a lot of work and could require changes that break popular 3rd party software (django-taggit comes to mind here). It would likely be possible to make the query construct work like in 1.5, too, but loud failure seems a lot better than a silently returning wrong data.

Marking as release blocker, docs need updating at least.

[pschmidt@…]

For anyone else who had a stack of production code depending on this apparently hidden / undocumented feature, I got around it by rewriting with subqueries, something like:

bookmarks = BookMark.objects.filter(url__icontains='django')
content_type = ContentType.objects.get(app_label='...', model='bookmark')
TaggedItem.objects.filter(object_id__in=bookmarks, content_type=content_type)

[Aymeric Augustin]

Given akaariai's explanation, I don't think this is a release blocker. Something that didn't work still doesn't work. This shouldn't prevent 1.7 from moving forwards.

[Tim Graham]

I'll mark this as fixed since #22207 added support for querying the reverse generic relation using the related_query_name option of GenericRelation.