bulk_create fails when related objects got PKs later

[Filip Sedlák]

This is a regression betwen Django 5.2 and 6.0.

Here's a short reproducer test that, when added to tests/bulk_create/tests.py fails with an AssertionError within bulk_create.

    @skipUnlessDBFeature("can_return_rows_from_bulk_insert")
    def test_pk_from_related_instance_saved_after_init(self):
        country = Country(name="Syldavia", iso_two_letter="SW")
        related = RelatedModel(country=country)
        country.save()
        RelatedModel.objects.bulk_create([related])
        self.assertEqual(related.country_id, country.pk)

Failure:

  File "/Users/krab/workspace/django/tests/bulk_create/tests.py", line 901, in test_pk_from_related_instance_saved_after_init
    RelatedModel.objects.bulk_create([related])
    ^^^^^^^^^^^^^^^
  File "/Users/krab/workspace/django/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
    ^^^^^^^
  File "/Users/krab/workspace/django/django/db/models/query.py", line 913, in bulk_create
    assert len(returned_columns) == len(objs_without_pk)
    ^^^

It looks like Django used to support this case intentionally, calling ._prepare_related_fields_for_save(operation_name="bulk_create") within bulk_create but a recent change breaks that.

I've never contributed to Django, but I'm willing to work on a fix if you're interested.

[Sarah Boyce]

Thank you for the test and ticket!
Regression in 7d9aab8da0a06787d649762a702e1a518d843a63 (refs #36260)

Marking as a release blocker as this was introduced in the latest feature release. If you think you have a fix, you're very welcome to raise a PR! You can review/test a PR from a different contributor. We will try to fix this before the planned release next week (Wednesday 5th August)

[Akshat Sparsh]

​PR

The full test suite passes on SQLite. The bulk_create tests also pass on PostgreSQL 18 and MariaDB 11.8.

[Jacob Walls]

Left notes on an additional test case we might want.

[Jacob Walls]

I restored the patch to the state suggested by Filip and agreed with Simon. (The detour I took into expression assignment is something we can more leisurely evaluate without backporting in #37239.)

[Jacob Walls <jacobtylerwalls@…>]

In c9ff757:

Fixed #37234 -- Fixed bulk_create() for late-saved related primary keys.

Objects with a primary key supplied by a late-saved related instance
were partitioned before their related fields were prepared. This
caused an assertion failure on backends returning rows from bulk
inserts.

Regression in 7d9aab8da0a06787d649762a702e1a518d843a63.

[Jacob Walls <jacobtylerwalls@…>]

In c800c8a:

[6.1.x] Fixed #37234 -- Fixed bulk_create() for late-saved related primary keys.

Objects with a primary key supplied by a late-saved related instance
were partitioned before their related fields were prepared. This
caused an assertion failure on backends returning rows from bulk
inserts.

Regression in 7d9aab8da0a06787d649762a702e1a518d843a63.

Backport of c9ff757a55392b1f50968eb89fe775f6155168d8 from main.

[Jacob Walls <jacobtylerwalls@…>]

In c81c6a9:

[6.0.x] Fixed #37234 -- Fixed bulk_create() for late-saved related primary keys.

Objects with a primary key supplied by a late-saved related instance
were partitioned before their related fields were prepared. This
caused an assertion failure on backends returning rows from bulk
inserts.

Regression in 7d9aab8da0a06787d649762a702e1a518d843a63.

Backport of c9ff757a55392b1f50968eb89fe775f6155168d8 from main.