Opened 17 years ago

Closed 16 years ago

#3141 closed defect (fixed)

can't use count() on querysets when extra() has been used with params.

Reported by: mrmachine <real dot human at mrmachine dot net> Owned by: deepak
Component: Database layer (models, ORM) Version: dev
Severity: normal Keywords: queryset extra params count, qs-rf-fixed
Cc: simon@…, real.human@…, heckj@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

there seems to be a bug in django/db/models/query.py when using extra() with params on a queryset. the queryset is generated correctly, i can access it, iterate through it, but i can't count() it.

>>> from django.contrib.auth.models import User
>>> u = User.objects.filter(pk=1).extra(select={'foo': '%s'}, params=['1'])
>>> u[0].foo
'1'
>>> u.count()
Traceback (most recent call last):
  File "<console>", line 1, in ?
  File "/Volumes/Singularity/Projects/django/django/db/models/query.py", line 202, in count
    cursor.execute("SELECT COUNT(*)" + sql, params)
  File "/Volumes/Singularity/Projects/django/django/db/backends/util.py", line 19, in execute
    self.db.queries.append({
TypeError: not all arguments converted during string formatting
>>> u = User.objects.filter(pk=1).extra(select={'foo': '1'})
>>> u[0].foo
'1'
>>> u.count()
1

Attachments (1)

extra_param_fix.diff (618 bytes ) - added by deepak <deep.thukral@…> 17 years ago.

Download all attachments as: .zip

Change History (9)

comment:1 by heckj@…, 17 years ago

Cc: heckj@… added

I just ran into a similiar situation this evening with the queryset not putting the arguments in the correct ordering, specifically when tacking on two "extra" sets to an otherwise kind of plain jane queryset. I'm trying to work it down to an easily reproducible case or example. In my case, it also appears to be related to using .count() at the tail end of the queryset chain.

If I pull back the queryset, force it into a list, then I can count it - but that sort of defeats the purpose of being able to use the count.

Ah - I narrowed it down to when it's using an extra(select=...) statement. In django/db/models/query.py around line 202, it's invoking:
cursor.execute("SELECT COUNT(*)" + sql, params)

The additional "select" param is added to the list, but the additional "select" component isn't, resulting in the "not all arguments converted" error.

Not sure how to nessecarily resolve this though.

comment:2 by Simon G. <dev@…>, 17 years ago

Triage Stage: UnreviewedAccepted

comment:3 by Malcolm Tredinnick, 17 years ago

Keywords: qs-rf-fixed added

by deepak <deep.thukral@…>, 17 years ago

Attachment: extra_param_fix.diff added

comment:4 by deepak <deep.thukral@…>, 17 years ago

Has patch: set
Needs tests: set
Owner: changed from nobody to deepak
Version: SVN

Main problem is that while logging sql, we are not handling extras in sql queries. I fixed this issue, but needs test.

comment:5 by amit.ramon@…, 16 years ago

The problem still exists with the latest svn version. It seems to a problem with using select with parameters in extra (extra where with parameters is working fine).

comment:6 by Malcolm Tredinnick, 16 years ago

(In [7340]) queryset-refactor: Fixed up extra(select=...) calls with parameters so that the
parameters are substituted in correctly in all cases. This introduces an extra
argument to extra() for this purpose; no alternative there.

Also fixed values() to work if you don't specify *all* the extra select aliases
in the values() call.

Refs #3141.

comment:7 by Simon Litchfield <simon@…>, 16 years ago

Cc: simon@… added

comment:8 by Malcolm Tredinnick, 16 years ago

Resolution: fixed
Status: newclosed

(In [7477]) Merged the queryset-refactor branch into trunk.

This is a big internal change, but mostly backwards compatible with existing
code. Also adds a couple of new features.

Fixed #245, #1050, #1656, #1801, #2076, #2091, #2150, #2253, #2306, #2400, #2430, #2482, #2496, #2676, #2737, #2874, #2902, #2939, #3037, #3141, #3288, #3440, #3592, #3739, #4088, #4260, #4289, #4306, #4358, #4464, #4510, #4858, #5012, #5020, #5261, #5295, #5321, #5324, #5325, #5555, #5707, #5796, #5817, #5987, #6018, #6074, #6088, #6154, #6177, #6180, #6203, #6658

Note: See TracTickets for help on using tickets.
Back to Top