Opened 4 years ago

Closed 3 years ago

#31496 closed Bug (fixed)

Combined queryset crash when chaining `values()` after `order_by()` with annotated constants.

Reported by: GardenLee Owned by: David Wobrock
Component: Database layer (models, ORM) Version: 3.0
Severity: Normal Keywords:
Cc: Hasan Ramezani, David Wobrock Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by David Wobrock)

When I tried query with 'annotate', 'union', 'order_by('annotate_field')' and 'values' at the same time, i got AttributeError('NoneType' object has no attribute 'split')

ex)
# 1. make 2 QuerySets

qs1 = Foo.objects.annotate(bar_val=Value(1, output_field=IntegerField())).filter(bar=False)
qs2 = Foo.objects.annotate(bar_val=Value(2, output_field=IntegerField())).filter(bar=True)

# 2. union

qs3 = qs1.union(qs2)

# 3. order_by annotate field

qs4 = qs3.order_by('bar_val')

# 4. get values

qs4.values('id')

In this case, I confirmed ('id', None) tuple passed by compiler.query.set_values() in django/db/models/sql/compiler.py

Change History (8)

comment:1 by Mariusz Felisiak, 4 years ago

Summary: When using 'annotate', 'union', 'order_by('annotate_field')' and 'values' at the same time, error occurredCombined queryset crash when chaining `values()` after `order_by()` with annotated constantants.
Triage Stage: UnreviewedAccepted

Thanks for this ticket. This code was changed in 2cbd3967e0a51eab993df89679046d25ec78baec, however previously it raised DatabaseError so IMO it's not a release blocker. We can consider backporting if patch will be straightforward.

comment:2 by Hasan Ramezani, 4 years ago

I take a look and here is my understanding:

In this case, when we order queryset by annotated value(bar_val) the src variable, has type <class 'django.db.models.functions.comparison.Cast'> and src.output_field.name is None.
So, add_select_col function adds None to values_select.

we should somehow add bar_val to the values_select.

@felixxm, If you have any workaround, I can prepare a patch.

comment:3 by Hasan Ramezani, 4 years ago

Cc: Hasan Ramezani added

comment:4 by David Wobrock, 4 years ago

Cc: David Wobrock added
Description: modified (diff)
Has patch: set
Owner: changed from nobody to David Wobrock
Status: newassigned
Summary: Combined queryset crash when chaining `values()` after `order_by()` with annotated constantants.Combined queryset crash when chaining `values()` after `order_by()` with annotated constants.

Hi,

Suggested patch: https://github.com/django/django/pull/13503
which probably needs some improvements here and there to be more concise and sustainable :)

Thanks!
David

comment:5 by David Wobrock, 4 years ago

Patch needs improvement: set

comment:6 by David Wobrock, 4 years ago

Patch needs improvement: unset

Took review into account and I changed the approach in the patch a bit :)

comment:7 by Mariusz Felisiak, 3 years ago

Triage Stage: AcceptedReady for checkin

comment:8 by Mariusz Felisiak <felisiak.mariusz@…>, 3 years ago

Resolution: fixed
Status: assignedclosed

In 464a4c0c:

Fixed #31496 -- Fixed QuerySet.values()/values_list() crash on combined querysets ordered by annotations.

Note: See TracTickets for help on using tickets.
Back to Top