Opened 2 years ago

Last modified 2 years ago

#33636 closed New feature

BulkProcessMixin on models.Model — at Version 2

Reported by: Myung Eui Yoon Owned by: Myung Eui Yoon
Component: Database layer (models, ORM) Version: 4.0
Severity: Normal Keywords: bulk, model
Cc: Triage Stage: Unreviewed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Myung Eui Yoon) 

        """
        class BulkyModel(models.Model, ModelBulkProcessMixin):
            id = models.BigAutoField(primary_key=True)
            name = models.CharField(max_length=10, null=False)
    
        With ModelBulkProcessMixin , We could minimize memory usage.
        Without ModelBulkProcessMixin, We shold maintain bulk array size up to 100_000
        or manually maintain arraysize up to batch_size like 10_000

        if len(chunked_list)>10_000:
            Model.objects.bulk_create(chunked_list)

        and check remain in list again at the end.
        
        if len(chunked_list)>0:
            Model.objects.bulk_create(chunked_list)
        """

        names = [f"name-{num}" for num in range(100_000)]

        with BulkyModel.gen_bulk_create(batch_size=10_000) as bulk:
            for name in names:
                bulk.add(BulkyModel(name=name))

        self.assertEqual(100_000, BulkyModel.objects.all().count())

​https://github.com/django/django/pull/15577

Change History (2)

comment:1 by Myung Eui Yoon, 2 years ago

Owner: changed from nobody to Myung Eui Yoon
Status: new → assigned

comment:2 by Myung Eui Yoon, 2 years ago

Description: modified (diff)
Has patch: set

Issued Pull Request
​https://github.com/django/django/pull/15577

Note: See TracTickets for help on using tickets.
Back to Top