Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#31420 closed Bug (fixed)

Using SimpleLazyObject with a nested subquery annotation fails.

Reported by: Jordan Ephron Owned by: Hasan Ramezani
Component: Database layer (models, ORM) Version: 3.0
Severity: Release blocker Keywords: simplelazyobject, queryset, subquery
Cc: Simon Charette Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Jordan Ephron)

Prior to 35431298226165986ad07e91f9d3aca721ff38ec it was possible to use a SimpleLazyObject in a queryset as demonstrated below. This new behavior appears to be a regression.


from django.contrib.auth.models import User
from django.db import models

class A(models.Model):

class B(models.Model):
    a = models.ForeignKey(A, on_delete=models.CASCADE)

class C(models.Model):
    owner = models.ForeignKey(User, on_delete=models.CASCADE)


from django.contrib.auth.models import User
from django.db.models import OuterRef, Subquery
from django.test import TestCase
from django.utils.functional import SimpleLazyObject

from ..models import A, B, C

class BugTestCase(TestCase):
    def test_bug(self):
        owner_user = (

        user = SimpleLazyObject(lambda: User.objects.create_user("testuser"))


Sorry for the somewhat arbitrary testcase, hopefully it's sufficient to repro this issue.


Traceback (most recent call last):
  File "/Users/u/PycharmProjects/django_debug/foo/tests/", line 20, in test_bug
  File "/Users/u/.virtualenvs/django_debug/src/django/django/db/models/", line 881, in filter
    return self._filter_or_exclude(False, *args, **kwargs)
  File "/Users/u/.virtualenvs/django_debug/src/django/django/db/models/", line 899, in _filter_or_exclude
    clone.query.add_q(Q(*args, **kwargs))
  File "/Users/u/.virtualenvs/django_debug/src/django/django/db/models/sql/", line 1297, in add_q
    clause, _ = self._add_q(q_object, self.used_aliases)
  File "/Users/u/.virtualenvs/django_debug/src/django/django/db/models/sql/", line 1325, in _add_q
    split_subq=split_subq, simple_col=simple_col,
  File "/Users/u/.virtualenvs/django_debug/src/django/django/db/models/sql/", line 1214, in build_filter
    condition = self.build_lookup(lookups, reffed_expression, value)
  File "/Users/u/.virtualenvs/django_debug/src/django/django/db/models/sql/", line 1123, in build_lookup
    lookup = lookup_class(lhs, rhs)
  File "/Users/u/.virtualenvs/django_debug/src/django/django/db/models/", line 20, in __init__
    self.rhs = self.get_prep_lookup()
  File "/Users/u/.virtualenvs/django_debug/src/django/django/db/models/", line 70, in get_prep_lookup
    return self.lhs.output_field.get_prep_value(self.rhs)
  File "/Users/u/.virtualenvs/django_debug/src/django/django/db/models/fields/", line 968, in get_prep_value
    return int(value)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'SimpleLazyObject'

Change History (10)

comment:1 by Jordan Ephron, 4 years ago

Description: modified (diff)

comment:2 by Jordan Ephron, 4 years ago

Description: modified (diff)

comment:3 by Mariusz Felisiak, 4 years ago

Cc: Simon Charette added
Severity: NormalRelease blocker
Summary: Regression when using SimpleLazyObject in a subqueryUsing SimpleLazyObject with a nested subquery annotation fails.
Triage Stage: UnreviewedAccepted

I agree that this behavior was changed in 35431298226165986ad07e91f9d3aca721ff38ec, but you can easily create a test with SimpleLazyObject() (which is not a public API) and Subquery() that always fails with TypeError, e.g.

def test_subquery_filter_by_lazy(self):
    test_user = SimpleLazyObject(lambda: get_user_model().objects.get(username='test'))
    qs = Company.objects.annotate(
    self.assertEqual(qs.get(), self.foobar_ltd)

This is somehow related with nested subqueries.

comment:4 by Simon Charette, 4 years ago

Alright so here's what's happening here.

When the most outer filter(owner_user=user) call is made the lookup logic tries to resolve owner_user's output_field in order to get_lookup('exact') on it. owner_user points to Subquery(owner_user) and it's .output_field is self.query.output_field. Since .values('owner_user') refers to Subquery(C.objects.values('owner')) which refers to self.query.output_field which refers to a column mapping to a Col referencing C.owner the actual outer most filter(owner_user=user) left hand side output field is whatever sql.Query.output_field returns for a selected field.

This happens to be Col.field

Now Col.field is actually Expression.field which is actually an alias for .output_field (not sure why) but in order to allow the special related field lookup conversion of SimpleLazyObject to model instance (e.g. needed for stuff like .filter(user=request.user) where request.user is a SimpleLazyObject) it's the that should be used as output_field and not Col.field.

  • django/db/models/sql/

    diff --git a/django/db/models/sql/ b/django/db/models/sql/
    index 78c4f47b5b..8ad9d139f9 100644
    a b class Query(BaseExpression):  
    233233    @property
    234234    def output_field(self):
    235235        if len( == 1:
    236             return[0].field
     236            return[0].target
    237237        elif len(self.annotation_select) == 1:
    238238            return next(iter(self.annotation_select.values())).output_field

The suite seems to agree from my local testing.

Last edited 4 years ago by Simon Charette (previous) (diff)

comment:5 by Mariusz Felisiak, 4 years ago

Confirmed. Thanks, Simon! It also fixes issue that never worked (see comment).

comment:6 by Hasan Ramezani, 4 years ago

Owner: changed from nobody to Hasan Ramezani
Status: newassigned

comment:7 by Jordan Ephron, 4 years ago

Thanks for the quick triage folks,

comment:8 by Hasan Ramezani, 4 years ago

Has patch: set

comment:9 by Mariusz Felisiak <felisiak.mariusz@…>, 4 years ago

Resolution: fixed
Status: assignedclosed

In 4237050:

Fixed #31420 -- Fixed crash when filtering subquery annotation against a SimpleLazyObject.

Thanks Simon Charette for the solution and analysis.

comment:10 by Mariusz Felisiak <felisiak.mariusz@…>, 4 years ago

In 22a2e97f:

[3.0.x] Fixed #31420 -- Fixed crash when filtering subquery annotation against a SimpleLazyObject.

Thanks Simon Charette for the solution and analysis.

Backport of 4237050684427db45ea834fe89d9e11c0520201e from master

Note: See TracTickets for help on using tickets.
Back to Top