Bad cascading leads to non-deterministic IntegrityError

[Marc Aymerich]

I've spent all day with a bug on my application that seems to be a bug on Django's ORM delete on cascade resolution order.

Moreover because the order in which related objects are deleted by the ORM is non-deterministic (changes every time you start the interpreter) it makes even harder to track down.

So far I've been able to reproduce the problem with a few models:

from django.db import models
from django.contrib.contenttypes.fields import GenericForeignKey, GenericRelation
from django.contrib.contenttypes.models import ContentType
from django.db.models.signals import post_delete
from django.dispatch import receiver

class Resource(models.Model):
    content_type = models.ForeignKey(ContentType)
    object_id = models.PositiveIntegerField()
    content_object = GenericForeignKey()

class Account(models.Model):
    name = models.CharField(max_length=10)
    resources = GenericRelation(Resource)

class Order(models.Model):
    account = models.ForeignKey(Account)
    content_type = models.ForeignKey(ContentType)
    object_id = models.PositiveIntegerField()
    content_object = GenericForeignKey()

class MetricStorage(models.Model):
    order = models.ForeignKey(Order)

@receiver(post_delete, dispatch_uid="orders.cancel_orders")
def cancel_orders(sender, **kwargs):
    instance = kwargs['instance']
    print('delete', sender, instance, instance.pk)
    if sender == Resource:
        related = instance.content_object
        if related:
            ct = ContentType.objects.get_for_model(related)
            order = Order.objects.get(content_type=ct, object_id=related.pk)
            order.metricstorage_set.create()
            order.save()
        else:
            print('related is None')

And this test code

from test.models import Account, Resource, Order

account = Account.objects.create(name='John')
resource = Resource.objects.create(content_object=account)
order = Order.objects.create(account=account, content_object=account)
account.delete()

In order to properly test this you should make tries restarting the interpreter a handful of times, and then you'll see two different results.

This that I consider correct:

delete <class 'test.models.Order'> Order object 384
delete <class 'test.models.Account'> Account object 124
delete <class 'test.models.Resource'> Resource object 307
related is None

And the IntegrityError which I consider to be a bug

delete <class 'test.models.Resource'> Resource object 311
delete <class 'test.models.Order'> Order object 388
delete <class 'test.models.Account'> Account object 128
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/base.py", line 872, in delete
    collector.delete()
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/deletion.py", line 314, in delete
    sender=model, instance=obj, using=self.using
  File "/usr/local/lib/python3.4/dist-packages/django/db/transaction.py", line 232, in __exit__
    connection.commit()
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 173, in commit
    self._commit()
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 142, in _commit
    return self.connection.commit()
  File "/usr/local/lib/python3.4/dist-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python3.4/dist-packages/django/utils/six.py", line 658, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 142, in _commit
    return self.connection.commit()
django.db.utils.IntegrityError: insert or update on table "test_metricstorage" violates foreign key constraint "test_metricstorage_order_id_730c757c66b8c627_fk_test_order_id"
DETAIL:  Key (order_id)=(388) is not present in table "test_order".

Notice how the order in which related objects are deleted is different, I believe this to be the source of the problem.

I've noticed this problem on 1.7 and now 1.8, not tested with master.

[Tim Graham]

I could also reproduce on master with your example code.

[sww]

Looks like this behavior's been here a while - looks like commit 272de9eb6baad45abec029aae92c2b7d9478c841 introduced the bug, so it's been a bug since 1.6.

[Laura Feier]

​PR

Tests pass under SQLite, MySQL and PostgreSQL

[Tim Graham <timograham@…>]

In 7862cbda:

Fixed #24576 -- Made deletion of related objects deterministic.