Opened 11 years ago
Last modified 8 years ago
#21160 closed Bug
in_bulk() fails on SQLite when passing more than 999 ids — at Initial Version
| Reported by: | Andrei Picus | Owned by: | nobody |
|---|---|---|---|
| Component: | Database layer (models, ORM) | Version: | dev |
| Severity: | Normal | Keywords: | in_bulk sqlite 999 1000 |
| Cc: | Triage Stage: | Ready for checkin | |
| Has patch: | yes | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description
As we all know, SQLite doesn't allow more than 999 variables in a query, failing with the 'too many SQL variables' error if you pass in more. This is a problem when using the in_bulk() method.
I've looked through the code and found out that all backends define a bulk_batch_size ops feature and it is set to 999 in the SQLite backend.
https://github.com/django/django/blob/master/django/db/backends/sqlite3/base.py#L140
This ops feature is only used in _batched_insert. I've written a patch that makes use of it in the in_bulk method as well. You can check it here
[link pending]
I've also written two tests that test passing in a large number of ids and also the efficiency in this case. The efficiency test runs only on databases that don't support 1000 variables in a query.
Any comments are welcome.