Code

Opened 4 years ago

Closed 4 years ago

Last modified 3 years ago

#12608 closed (fixed)

Inconsistent results from values and values_list when using annotate

Reported by: mattmcc Owned by: coleifer
Component: Database layer (models, ORM) Version: master
Severity: Keywords: values_list annotate
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

To group users by email and get a count, one might do:

>>> User.objects.values('email').annotate(Count('email'))
[{'email__count': 2, 'email': u'user@example.com'}]

However, values_list discards the annotated value:

>>> User.objects.values_list('email').annotate(Count('email'))
[(u'user@example.com',)]

Attachments (2)

12608.patch (1.2 KB) - added by coleifer 4 years ago.
12608_fix.diff (804 bytes) - added by coleifer 4 years ago.

Download all attachments as: .zip

Change History (11)

comment:1 Changed 4 years ago by mattmcc

  • Component changed from Uncategorized to Database layer (models, ORM)
  • Keywords values_list annotate added
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

comment:2 Changed 4 years ago by russellm

  • Triage Stage changed from Unreviewed to Accepted

comment:3 Changed 4 years ago by coleifer

  • Has patch set

I patched the ValuesList iterator to include aggregates in the fields if specified:

diff --git a/django/db/models/query.py b/django/db/models/query.py
index 8cb3dbe..31b3333 100644
--- a/django/db/models/query.py
+++ b/django/db/models/query.py
@@ -954,7 +954,8 @@ class ValuesListQuerySet(ValuesQuerySet):
             # If a field list has been specified, use it. Otherwise, use the
             # full list of fields, including extras and aggregates.
             if self._fields:
-                fields = self._fields
+                fields = list(self._fields) + filter(lambda f: f not in self._fields,
+                                                     aggregate_names)
             else:
                 fields = names
 
diff --git a/tests/modeltests/aggregation/models.py b/tests/modeltests/aggregation/models.py
index 9ed638e..0e8d881 100644
--- a/tests/modeltests/aggregation/models.py
+++ b/tests/modeltests/aggregation/models.py
@@ -362,4 +362,7 @@ True
 >>> Book.objects.filter(pk=1).annotate(mean_age=Avg('authors__age')).values_list('mean_age', flat=True)
 [34.5]
 
+>>> Book.objects.values_list('price').annotate(count=Count('price')).order_by('-count', 'price')
+[(Decimal('29.69'), 2), (Decimal('23.09'), 1), (Decimal('30'), 1), (Decimal('75'), 1), (Decimal('82.8'), 1)]
+
 """}
In [1]: from django.contrib.auth.models import User

In [2]: from django.db.models import Count

In [3]: User.objects.values_list('email').annotate(Count('email'))
Out[3]: [(u'user@example.com', 2)]

In [4]: User.objects.values('email').annotate(Count('email'))
Out[4]: [{'email__count': 2, 'email': u'user@example.com'}]

In [5]: User.objects.values_list('email')
Out[5]: [(u'user@example.com',), (u'user@example.com',)]

Changed 4 years ago by coleifer

comment:4 Changed 4 years ago by coleifer

  • Owner changed from nobody to coleifer
  • Status changed from new to assigned

comment:5 Changed 4 years ago by Alex

  • Triage Stage changed from Accepted to Ready for checkin

comment:6 Changed 4 years ago by jbronn

  • Resolution set to fixed
  • Status changed from assigned to closed

(In [12505]) Fixed #12608 -- No longer return inconsistent results when using values and values_list in conjunction with annotate. Thanks, Charlie Leifer.

comment:7 Changed 4 years ago by coleifer

Updated doctest to work with postgresql. Tested & working now.

Changed 4 years ago by coleifer

comment:8 Changed 4 years ago by jbronn

(In [12613]) [1.1.X] Fixed #12608 -- No longer return inconsistent results when using values and values_list in conjunction with annotate. Thanks, Charlie Leifer.

Backport of r12505 from trunk.

comment:9 Changed 3 years ago by jacob

  • milestone 1.2 deleted

Milestone 1.2 deleted

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.