Opened 5 months ago

Last modified 2 months ago

#36416 closed Bug

id_list argument to in_bulk() does not account for composite primary keys when batching — at Version 2

Reported by: Jacob Walls Owned by: Jacob Walls
Component: Database layer (models, ORM) Version: 5.2
Severity: Release blocker Keywords:
Cc: Simon Charette, Csirmaz Bendegúz Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Jacob Walls)

The id_list argument to in_bulk() divides large lists into batches, but similar to #36118, it did not account for composite primary keys, leading to errors like this on SQLite:

django.db.utils.OperationalError: too many SQL variables

#36118 mentioned that other uses like this remained to be audited.

The id_list argument is tested, but the batching was not covered.

Failing test (I may adjust this in the PR to run faster by mocking a lower query param limit, but this shows the OperationalError):

  • tests/composite_pk/tests.py

    diff --git a/tests/composite_pk/tests.py b/tests/composite_pk/tests.py
    index 91cbee0635..ba290a796f 100644
    a b class CompositePKTests(TestCase):  
    147147        result = Comment.objects.in_bulk([self.comment.pk])
    148148        self.assertEqual(result, {self.comment.pk: self.comment})
    149149
     150    def test_in_bulk_batching(self):
     151        num_requiring_batching = (connection.features.max_query_params // 2) + 1
     152        comments = [
     153            Comment(id=i, tenant=self.tenant)
     154            for i in range(2, num_requiring_batching + 1)
     155        ]
     156        Comment.objects.bulk_create(comments)
     157        id_list = list(Comment.objects.values_list("pk", flat=True))
     158        with self.assertNumQueries(2):
     159            comment_dict = Comment.objects.in_bulk(id_list=id_list)
     160        self.assertQuerySetEqual(comment_dict, id_list)
     161
    150162    def test_iterator(self):
    151163        """
    152164        Test the .iterator() method of composite_pk models.

Change History (2)

comment:1 by Jacob Walls, 5 months ago

Has patch: set

comment:2 by Jacob Walls, 5 months ago

Description: modified (diff)
Note: See TracTickets for help on using tickets.
Back to Top