Opened 6 months ago

Last modified 3 months ago

#29396 assigned Bug

Using date__year comparisons together with OuterRef causes an IndexError

Reported by: Dmitry Shachnev Owned by: Windson yang
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

With the following models:

class ModelWithDate(models.Model):
    id = models.AutoField(primary_key=True)
    date = models.DateField()

class ModelWithYear(models.Model):
    id = models.AutoField(primary_key=True)
    year = models.IntegerField()
    date_ref = models.ForeignKey(to=ModelWithDate, on_delete=models.CASCADE)

the following code:

>>> dates = ModelWithDate.objects.filter(date__year__gte=OuterRef("year"))
>>> dates_subq = Subquery(dates.values("id"))
>>> ModelWithYear.objects.filter(date_ref__in=dates_subq)

causes an exception:

Traceback (most recent call last):
  ...
  File "/usr/lib/python3/dist-packages/django/db/models/lookups.py", line 90, in process_rhs
    sql, params = compiler.compile(value)
  File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 391, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/usr/lib/python3/dist-packages/django/db/models/expressions.py", line 1041, in as_sql
    template_params['subquery'], sql_params = self.queryset.query.get_compiler(connection=connection).as_sql()
  File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 459, in as_sql
    where, w_params = self.compile(self.where) if self.where is not None else ("", [])
  File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 391, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/usr/lib/python3/dist-packages/django/db/models/sql/where.py", line 80, in as_sql
    sql, params = compiler.compile(child)
  File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 391, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/usr/lib/python3/dist-packages/django/db/models/lookups.py", line 523, in as_sql
    start, finish = self.year_lookup_bounds(connection, rhs_params[0])
IndexError: list index out of range

The same happens if I create an annotation using ExtractYear and use it directly.

A small test project is attached. The bug can be reproduced with “DJANGO_SETTINGS_MODULE=settings python3 ./test.py”. Tested with Django 2.0.5 and 1.11.13.

Attachments (1)

test-project.tar.gz (796 bytes) - added by Dmitry Shachnev 6 months ago.
Test project to reproduce the bug

Download all attachments as: .zip

Change History (7)

Changed 6 months ago by Dmitry Shachnev

Attachment: test-project.tar.gz added

Test project to reproduce the bug

comment:1 Changed 6 months ago by Dmitry Shachnev

I should add that it happens only with gte/lte comparisons. If I replace date__year__gte with date__year__exact, then the code works as expected.

comment:2 Changed 6 months ago by Tim Graham

Triage Stage: UnreviewedAccepted
Version: master

Reproduced on master at 9c4ea63e878c053600c284e32d5f32d27a59b63a.

comment:3 Changed 6 months ago by Windson yang

Owner: changed from nobody to Windson yang
Status: newassigned

comment:4 Changed 6 months ago by Windson yang

Please tell me if I'm doing wrong, Just like YearExact, We should adding the same try-except block to handle when rhs_params[0] doesn't exist. In the except part, should we just raise an error like below? Or we have to overwrite GreaterThanOrEqual and LessThanOrEqual class?

    raise ValueError(
                    'The QuerySet value for an exact lookup must be limited to '
                    'one result using slicing.'
                )

comment:6 Changed 3 months ago by Tim Graham

Has patch: set
Patch needs improvement: set
Note: See TracTickets for help on using tickets.
Back to Top