in_bulk() fails on SQLite when passing more than 999 ids
|Reported by:||NiGhTTraX||Owned by:||nobody|
|Component:||Database layer (models, ORM)||Version:||master|
|Severity:||Normal||Keywords:||in_bulk sqlite 999 1000|
|Has patch:||yes||Needs documentation:||no|
|Needs tests:||no||Patch needs improvement:||yes|
Description (last modified by EvilDMP)
As we all know, SQLite doesn't allow more than 999 variables in a query, failing with the 'too many SQL variables' error if you pass in more. This is a problem when using the in_bulk() method.
I've looked through the code and found out that all backends define a bulk_batch_size ops feature and it is set to 999 in the SQLite backend.
This ops feature is only used in _batched_insert. I've written a patch that makes use of it in the in_bulk method as well. You can check it here: https://github.com/django/django/pull/1679
I've also written two tests that test passing in a large number of ids and also the efficiency in this case. The efficiency test runs only on databases that don't support 1000 variables in a query.
Any comments are welcome.
Change History (6)
comment:1 Changed 2 years ago by NiGhTTraX
- Needs documentation unset
- Needs tests unset
- Patch needs improvement unset