django 1.7.1 defer() throws AttributeError when using related_name

[Farhan Ahammed]

I have attached a simple example project, with the following Models defined:

class ModelA(models.Model):
    name = models.CharField(max_length=255)

class ModelB(models.Model):
    name = models.CharField(max_length=255)
    model_a = models.ForeignKey(ModelA, related_name='model_b')

Entering the following code (in a Django shell) works fine:

ModelB.objects.filter(name='').select_related('model_a').defer('model_a__name')

But when I run the following:

ModelA.objects.filter(name='').select_related('model_b').defer('model_b__name')

I get this error message:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/fahammed/Documents/django_defer_bug/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 116, in __repr__
    data = list(self[:REPR_OUTPUT_SIZE + 1])
  File "/home/fahammed/Documents/django_defer_bug/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 141, in __iter__
    self._fetch_all()
  File "/home/fahammed/Documents/django_defer_bug/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 966, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/home/fahammed/Documents/django_defer_bug/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 222, in iterator
    only_load = self.query.get_loaded_field_names()
  File "/home/fahammed/Documents/django_defer_bug/venv/local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 1821, in get_loaded_field_names
    self.deferred_to_data(collection, self.get_loaded_field_names_cb)
  File "/home/fahammed/Documents/django_defer_bug/venv/local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 596, in deferred_to_data
    cur_model = source.rel.to
AttributeError: 'RelatedObject' object has no attribute 'rel'

Looks like defer doesn't work when you specify the related table using its related_name.

[Farhan Ahammed]

Example project to run/test the bug

[Tim Graham]

This seems to be a regression present since 1.6 -- bisected to 70679243d1786e03557c28929f9762a119e3ac14. Test for Django's test suite attached.

[Anssi Kääriäinen]

The reason pre-1.6 didn't raise exceptions was that the chunked reads implementation swallowed exceptions. I don't think there is a regression here, we just get an exception instead of silently returning empty list on error.

I tested this on Django 1.5 with the following command: [o for o in ModelB.objects.filter(name='').select_related('model_b').defer('model_b__name').iterator()]. This raises a different error (ModelB has no field named 'model_b'), but doesn't produce any results.

We might want to allow usage of related_name in this context, and even if we don't, users shouldn't get such an ugly error message.

[Farhan Ahammed]

@akaariai That's not the code that was causing the error message. From ModelB.objects.filter(name=''), you select the related objects from ModelA and then defer the ModelA.name value from the DB. So your command should probably be:

[o for o in ModelB.objects.filter(name='').select_related('model_a').defer('model_a__name').iterator()]

Note that ModelB has no field called model_b} but it does have a field called model_a.

[Anssi Kääriäinen]

I'm confused. Using [o for o in ModelA.objects.filter(name='').select_related('model_b').defer('model_b__name')] gives me RelatedObject has no attribute rel in 1.5. Just as it does using master. And, [o for o in ModelB.objects.filter(name='').select_related('model_a').defer('model_a__name').iterator()] works on master.

[Farhan Ahammed]

I think that's because the attribute model_a has been explicitly defined in the class definition of ModelB, whereas the class definition of ModelA does not have an attribute called model_b. Correct me if I'm wrong, but the ModelA.model_b is dynamically created at runtime (or loaded when the value is called)?

On another (possibly) similar note, calling the only() function:

ModelA.objects.filter(name='').select_related('model_b').only('model_b__name')

also returns the same error AttributeError: 'RelatedObject' object has no attribute 'rel', and the following line works fine:

ModelB.objects.filter(name='').select_related('model_a').only('model_a__name')

[Tim Graham]

The problem seems to be an invalid select_related() query. On Django 1.9:

>>> ModelA.objects.filter(name='').select_related('model_b').defer('model_b__name')
django.core.exceptions.FieldError: Invalid field name(s) given in select_related: 'model_b'. Choices are: (none)

You should be using prefech_related() when querying reverse foreign keys.