Opened 12 years ago
Closed 9 years ago
#19263 closed Bug (fixed)
Filtering __in a sliced queryset with a 0 limit raises an error
Reported by: | Owned by: | Marcin Biernat | |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | 1.4 |
Severity: | Normal | Keywords: | |
Cc: | timograham@… | Triage Stage: | Ready for checkin |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description (last modified by )
I've noticed that after upgrading to Django 1.4, __in
queries really don't like empty sets. Simple queries still work, like User.objects.filter(groups__in=[])
, but most failures I've seen are with Paginators. I think this is the minimum set to cause a DatabaseError, create any app, add a models.py with:
from django.db import models class Author(models.Model): pass class Book(models.Model): author = models.ForeignKey(Author) def crash(): from django.core.paginator import Paginator pages = Paginator(Author.objects.all(), 25) page = pages.page(1) books = Book.objects.filter(author__in=page.object_list) print books
calling crash() will cause this stack trace:
C:\Workspace\someproject\src\someproject\test.py in <module>() 6 7 books = Book.objects.filter(author__in=page.object_list) ----> 8 print books 9 C:\Dev\Python27\lib\site-packages\django\db\models\query.pyc in __repr__(self) 70 71 def __repr__(self): ---> 72 data = list(self[:REPR_OUTPUT_SIZE + 1]) 73 if len(data) > REPR_OUTPUT_SIZE: 74 data[-1] = "...(remaining elements truncated)..." C:\Dev\Python27\lib\site-packages\django\db\models\query.pyc in __len__(self) 85 self._result_cache = list(self.iterator()) 86 elif self._iter: ---> 87 self._result_cache.extend(self._iter) 88 if self._prefetch_related_lookups and not self._prefetch_done: 89 self._prefetch_related_objects() C:\Dev\Python27\lib\site-packages\django\db\models\query.pyc in iterator(self) 289 klass_info = get_klass_info(model, max_depth=max_depth, 290 requested=requested, only_load=only_load) --> 291 for row in compiler.results_iter(): 292 if fill_cache: 293 obj, _ = get_cached_row(row, index_start, db, klass_info, C:\Dev\Python27\lib\site-packages\django\db\models\sql\compiler.pyc in results_iter(self) 761 if self.query.select_for_update and transaction.is_managed(self.using): 762 transaction.set_dirty(self.using) --> 763 for rows in self.execute_sql(MULTI): 764 for row in rows: 765 if resolve_columns: C:\Dev\Python27\lib\site-packages\django\db\models\sql\compiler.pyc in execute_sql(self, result_type) 816 817 cursor = self.connection.cursor() --> 818 cursor.execute(sql, params) 819 820 if not result_type: C:\Dev\Python27\lib\site-packages\django\db\backends\util.pyc in execute(self, sql, params) 38 start = time() 39 try: ---> 40 return self.cursor.execute(sql, params) 41 finally: 42 stop = time() C:\Dev\Python27\lib\site-packages\django\db\backends\postgresql_psycopg2\base.pyc in execute(self, query, args) 50 def execute(self, query, args=None): 51 try: ---> 52 return self.cursor.execute(query, args) 53 except Database.IntegrityError, e: 54 raise utils.IntegrityError, utils.IntegrityError(*tuple(e)), sys.exc_info()[2] DatabaseError: syntax error at or near ")" LINE 1: ...ugtest_book" WHERE "bugtest_book"."author_id" IN () LIMIT 21
The SQL statement created is:
SELECT "bugtest_book"."id", "bugtest_book"."author_id" FROM "bugtest_book" WHERE "bugtest_book"."author_id" IN () LIMIT 21
Change History (20)
comment:1 by , 12 years ago
comment:2 by , 12 years ago
Component: | Uncategorized → Core (Other) |
---|---|
Description: | modified (diff) |
Triage Stage: | Unreviewed → Accepted |
Type: | Uncategorized → Bug |
I could reproduce the failure under PostgreSQL.
comment:3 by , 12 years ago
Owner: | changed from | to
---|---|
Status: | new → assigned |
comment:4 by , 12 years ago
Has patch: | set |
---|
This was a bug related to improper behaviour of querysets sliced with equal indices, which generated empty sql that caused errors when used in subqueries.
This query failed:
Book.objects.filter(author__in=Author.objects.empty()[0:0])
while this succeded:
Book.objects.filter(author__in=Author.objects.empty())
Instead of later returning empty sql string, a QuerySet with equal indices is now immediately converted to QuerySet.none().
comment:5 by , 12 years ago
Component: | Core (Other) → Database layer (models, ORM) |
---|
comment:8 by , 12 years ago
Fixed by checking for empty sql params
https://github.com/django/django/pull/813
comment:9 by , 12 years ago
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
comment:11 by , 12 years ago
Resolution: | fixed |
---|---|
Status: | closed → new |
Triage Stage: | Ready for checkin → Accepted |
This commit introduced test failures under all databases except SQLite.
If it cannot be fixed quickly, it should be reverted to avoid getting in the way of the sprints today.
comment:12 by , 12 years ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
comment:13 by , 12 years ago
Resolution: | fixed |
---|---|
Status: | closed → new |
Reverted in cbb9f629b88d97dd9a3f8d425fd56c8b80d7cddf, working on it.
comment:14 by , 12 years ago
Owner: | changed from | to
---|---|
Status: | new → assigned |
fixed and tested on sqlite, pgsql, oracle and mysql:
comment:15 by , 12 years ago
Owner: | changed from | to
---|
comment:16 by , 12 years ago
Triage Stage: | Accepted → Ready for checkin |
---|
comment:17 by , 11 years ago
Cc: | added |
---|---|
Patch needs improvement: | set |
Triage Stage: | Ready for checkin → Accepted |
Patch no longer applies cleanly after 97774429aeb54df4c09895c07cd1b09e70201f7d
comment:18 by , 9 years ago
Patch needs improvement: | unset |
---|---|
Summary: | DatabaseError on using empty Page .object_list in __in clause in a query → Filtering __in a sliced queryset with a 0 limit raises an error |
comment:19 by , 9 years ago
Triage Stage: | Accepted → Ready for checkin |
---|
I've traced the change back, but it doesn't seem to be introduced at 1.4. On 1.2, this works:
On 1.3.4:
It's also database dependent, I couldn't reproduce the issue with SQLite.