Opened 6 years ago

Closed 5 years ago

Last modified 5 years ago

#29396 closed Bug (fixed)

The __year lookp crashes with IndexError when passed a non-direct values/expression.

Reported by: Dmitry Shachnev Owned by: Windson yang
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

With the following models:

class ModelWithDate(models.Model):
    id = models.AutoField(primary_key=True)
    date = models.DateField()

class ModelWithYear(models.Model):
    id = models.AutoField(primary_key=True)
    year = models.IntegerField()
    date_ref = models.ForeignKey(to=ModelWithDate, on_delete=models.CASCADE)

the following code:

>>> dates = ModelWithDate.objects.filter(date__year__gte=OuterRef("year"))
>>> dates_subq = Subquery(dates.values("id"))
>>> ModelWithYear.objects.filter(date_ref__in=dates_subq)

causes an exception:

Traceback (most recent call last):
  ...
  File "/usr/lib/python3/dist-packages/django/db/models/lookups.py", line 90, in process_rhs
    sql, params = compiler.compile(value)
  File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 391, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/usr/lib/python3/dist-packages/django/db/models/expressions.py", line 1041, in as_sql
    template_params['subquery'], sql_params = self.queryset.query.get_compiler(connection=connection).as_sql()
  File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 459, in as_sql
    where, w_params = self.compile(self.where) if self.where is not None else ("", [])
  File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 391, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/usr/lib/python3/dist-packages/django/db/models/sql/where.py", line 80, in as_sql
    sql, params = compiler.compile(child)
  File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 391, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/usr/lib/python3/dist-packages/django/db/models/lookups.py", line 523, in as_sql
    start, finish = self.year_lookup_bounds(connection, rhs_params[0])
IndexError: list index out of range

The same happens if I create an annotation using ExtractYear and use it directly.

A small test project is attached. The bug can be reproduced with “DJANGO_SETTINGS_MODULE=settings python3 ./test.py”. Tested with Django 2.0.5 and 1.11.13.

Attachments (1)

test-project.tar.gz (796 bytes ) - added by Dmitry Shachnev 6 years ago.
Test project to reproduce the bug

Download all attachments as: .zip

Change History (10)

by Dmitry Shachnev, 6 years ago

Attachment: test-project.tar.gz added

Test project to reproduce the bug

comment:1 by Dmitry Shachnev, 6 years ago

I should add that it happens only with gte/lte comparisons. If I replace date__year__gte with date__year__exact, then the code works as expected.

comment:2 by Tim Graham, 6 years ago

Triage Stage: UnreviewedAccepted
Version: master

Reproduced on master at 9c4ea63e878c053600c284e32d5f32d27a59b63a.

comment:3 by Windson yang, 6 years ago

Owner: changed from nobody to Windson yang
Status: newassigned

comment:4 by Windson yang, 6 years ago

Please tell me if I'm doing wrong, Just like YearExact, We should adding the same try-except block to handle when rhs_params[0] doesn't exist. In the except part, should we just raise an error like below? Or we have to overwrite GreaterThanOrEqual and LessThanOrEqual class?

    raise ValueError(
                    'The QuerySet value for an exact lookup must be limited to '
                    'one result using slicing.'
                )

comment:6 by Tim Graham, 6 years ago

Has patch: set
Patch needs improvement: set

comment:7 by Simon Charette, 5 years ago

Summary: Using date__year comparisons together with OuterRef causes an IndexErrorThe __year lookp crashes with IndexError when passed a non-direct values/expression.

As I commented on the PR the bug has little to do with subquery usage; __year lookup simply never supported non-direct values and OuterRef happens to implement the expression API in this particular case.

Both this issue and #30494 which was for the year__exact case are handled by https://github.com/django/django/pull/11393.

comment:8 by Mariusz Felisiak <felisiak.mariusz@…>, 5 years ago

Resolution: fixed
Status: assignedclosed

In 2b582a7:

Fixed #29396 -- Added indirect values support to year lookups.

Thanks Windson Yang for the initial patch.

comment:9 by Mariusz Felisiak <felisiak.mariusz@…>, 5 years ago

In 514104cf:

Refs #29396, #30494 -- Reduced code duplication in year lookups.

Note: See TracTickets for help on using tickets.
Back to Top