Opened 11 years ago
Last modified 8 years ago
#21160 closed Bug
in_bulk() fails on SQLite when passing more than 999 ids — at Version 3
Reported by: | Andrei Picus | Owned by: | nobody |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | dev |
Severity: | Normal | Keywords: | in_bulk sqlite 999 1000 |
Cc: | Triage Stage: | Ready for checkin | |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description (last modified by )
As we all know, SQLite doesn't allow more than 999 variables in a query, failing with the 'too many SQL variables' error if you pass in more. This is a problem when using the in_bulk() method.
I've looked through the code and found out that all backends define a bulk_batch_size ops feature and it is set to 999 in the SQLite backend.
https://github.com/django/django/blob/master/django/db/backends/sqlite3/base.py#L140
This ops feature is only used in _batched_insert. I've written a patch that makes use of it in the in_bulk method as well. You can check it here: https://github.com/django/django/pull/1679
I've also written two tests that test passing in a large number of ids and also the efficiency in this case. The efficiency test runs only on databases that don't support 1000 variables in a query.
Any comments are welcome.
Change History (3)
comment:1 by , 11 years ago
comment:2 by , 11 years ago
Severity: | Release blocker → Normal |
---|
comment:3 by , 11 years ago
Description: | modified (diff) |
---|
Pull request: https://github.com/django/django/pull/1679
For the love of me I can't find the way to edit the original description.