Opened 14 months ago

Closed 14 months ago

Last modified 14 months ago

#34570 closed Bug (fixed)

QuerySet.defer() raises an AttributeError when the field is ManyToManyField or GenericForeignKey

Reported by: Paco Martínez Owned by: Simon Charette
Component: Database layer (models, ORM) Version: 4.2
Severity: Release blocker Keywords: contenttypes, defer, manytomanyfield, genericforeignkey
Cc: Simon Charette Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

In Version 4.2.1

Hello, when I try to defer a field that is a ManyToManyField or a GenericForeignKey it raises an AttributteError.

Having these three models:

from django.db import models
from django.contrib.contenttypes.fields import GenericForeignKey, ContentType

class ModelA(models.Model):
    name = models.CharField(max_length=100)
  

class ModelB(models.Model):
    name = models.CharField(max_length=100)
    model_a = models.ManyToManyField('ModelA')
    

class ModelC(models.Model):
    name = models.CharField(max_length=100)
    content_type = models.ForeignKey(ContentType, null=True, on_delete=models.CASCADE)
    object_id = models.PositiveIntegerField(null=True)
    generic_model = GenericForeignKey('content_type', 'object_id')

Traceback from a ManyToManyField:

>>> ModelB.objects.defer('model_a')
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 374, in __repr__
    data = list(self[: REPR_OUTPUT_SIZE + 1])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 398, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1881, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 91, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1547, in execute_sql
    sql, params = self.as_sql()
                  ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 734, in as_sql
    extra_select, order_by, group_by = self.pre_sql_setup(
                                       ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 84, in pre_sql_setup
    self.setup_query(with_col_aliases=with_col_aliases)
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 73, in setup_query
    self.select, self.klass_info, self.annotation_col_map = self.get_select(
                                                            ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 256, in get_select
    select_mask = self.query.get_select_mask()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/query.py", line 772, in get_select_mask
    return self._get_defer_select_mask(opts, mask)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/query.py", line 728, in _get_defer_select_mask
    field = opts.get_field(field_name).field
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ManyToManyField' object has no attribute 'field'

Traceback from defering a GenericForeignKey:

>>> ModelC.objects.defer('generic_model')
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 374, in __repr__
    data = list(self[: REPR_OUTPUT_SIZE + 1])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 398, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1881, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 91, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1547, in execute_sql
    sql, params = self.as_sql()
                  ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 734, in as_sql
    extra_select, order_by, group_by = self.pre_sql_setup(
                                       ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 84, in pre_sql_setup
    self.setup_query(with_col_aliases=with_col_aliases)
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 73, in setup_query
    self.select, self.klass_info, self.annotation_col_map = self.get_select(
                                                            ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 256, in get_select
    select_mask = self.query.get_select_mask()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/query.py", line 772, in get_select_mask
    return self._get_defer_select_mask(opts, mask)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/query.py", line 728, in _get_defer_select_mask
    field = opts.get_field(field_name).field
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'GenericForeignKey' object has no attribute 'field'

PD:

In version 4.1.9 it is working without any error, but in versions 4.2.0 and 4.2.1 raises the AttributeError

Change History (8)

comment:1 by Mariusz Felisiak, 14 months ago

Cc: Simon Charette added
Severity: NormalRelease blocker
Triage Stage: UnreviewedAccepted

Thanks for the report!

Regression in b3db6c8dcb5145f7d45eff517bcd96460475c879.
Reproduced at 0ec60661e61b153e6bcec64649b1b7f524eb3e18.

comment:2 by Simon Charette, 14 months ago

Owner: changed from nobody to Simon Charette
Status: newassigned

In version 4.1.9 it is working without any error, but in versions 4.2.0 and 4.2.1 raises the AttributeError

Could you elaborate on that? Many-to-many fields and generic foreign keys are not deferrable (they are virtual and do not map to columns) so the previous code was simply swallowing invalid values passed to defer.

I guess that raising a more detailed exception would still require a deprecation period and that it would be better to perform this validation at defer/only instead of compilation time anyway so our only option here is to silence cases where opts.get_field(field_name) returns something that doesn't have a .field property.

comment:3 by Simon Charette, 14 months ago

Has patch: set

comment:4 by Mariusz Felisiak, 14 months ago

Triage Stage: AcceptedReady for checkin

comment:5 by Mariusz Felisiak <felisiak.mariusz@…>, 14 months ago

Resolution: fixed
Status: assignedclosed

In 99e5dff7:

Fixed #34570 -- Silenced noop deferral of many-to-many and GFK.

While deferring many-to-many and GFK has no effect, the previous
implementation of QuerySet.defer() ignore them instead of crashing.

Regression in b3db6c8dcb5145f7d45eff517bcd96460475c879.

Thanks Paco Martínez for the report.

comment:6 by Mariusz Felisiak <felisiak.mariusz@…>, 14 months ago

In 201d29b:

[4.2.x] Fixed #34570 -- Silenced noop deferral of many-to-many and GFK.

While deferring many-to-many and GFK has no effect, the previous
implementation of QuerySet.defer() ignore them instead of crashing.

Regression in b3db6c8dcb5145f7d45eff517bcd96460475c879.

Thanks Paco Martínez for the report.

Backport of 99e5dff737cd20b12d060e4794e097063b61ec40 from main

in reply to:  2 comment:7 by Paco Martínez, 14 months ago

Replying to Simon Charette:

In version 4.1.9 it is working without any error, but in versions 4.2.0 and 4.2.1 raises the AttributeError

Could you elaborate on that? Many-to-many fields and generic foreign keys are not deferrable (they are virtual and do not map to columns) so the previous code was simply swallowing invalid values passed to defer.

I guess that raising a more detailed exception would still require a deprecation period and that it would be better to perform this validation at defer/only instead of compilation time anyway so our only option here is to silence cases where opts.get_field(field_name) returns something that doesn't have a .field property.

Well, the more I read about defer() and what it was for, the more I wondered that we were deferring ManyToMany and GenericForeignKey fields. Thanks for confirming that it was useless and that it just swallowed it without taking any action.

Thank you very much to both of you for your answers and for your quickness in solving it.

comment:8 by Mariusz Felisiak <felisiak.mariusz@…>, 14 months ago

In d9e7018:

Refs #34570 -- Added extra tests for QuerySet.only() noops.

Note: See TracTickets for help on using tickets.
Back to Top