Opened 11 years ago

Last modified 7 years ago

#21160 closed Bug

in_bulk() fails on SQLite when passing more than 999 ids — at Version 3

Reported by: Andrei Picus Owned by: nobody
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords: in_bulk sqlite 999 1000
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Daniele Procida)

As we all know, SQLite doesn't allow more than 999 variables in a query, failing with the 'too many SQL variables' error if you pass in more. This is a problem when using the in_bulk() method.

I've looked through the code and found out that all backends define a bulk_batch_size ops feature and it is set to 999 in the SQLite backend.
https://github.com/django/django/blob/master/django/db/backends/sqlite3/base.py#L140

This ops feature is only used in _batched_insert. I've written a patch that makes use of it in the in_bulk method as well. You can check it here: https://github.com/django/django/pull/1679

I've also written two tests that test passing in a large number of ids and also the efficiency in this case. The efficiency test runs only on databases that don't support 1000 variables in a query.

Any comments are welcome.

Change History (3)

comment:1 by Andrei Picus, 11 years ago

Pull request: https://github.com/django/django/pull/1679

For the love of me I can't find the way to edit the original description.

comment:2 by Tim Graham, 11 years ago

Severity: Release blockerNormal

comment:3 by Daniele Procida, 11 years ago

Description: modified (diff)
Note: See TracTickets for help on using tickets.
Back to Top