Opened 9 years ago

Closed 8 years ago

#24576 closed Bug (fixed)

Bad cascading leads to non-deterministic IntegrityError

Reported by: Marc Aymerich Owned by: nobody
Component: Database layer (models, ORM) Version: 1.8
Severity: Normal Keywords:
Cc: feierlaura10@… Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Tim Graham)

I've spent all day with a bug on my application that seems to be a bug on Django's ORM delete on cascade resolution order.

Moreover because the order in which related objects are deleted by the ORM is non-deterministic (changes every time you start the interpreter) it makes even harder to track down.

So far I've been able to reproduce the problem with a few models:

from django.db import models
from django.contrib.contenttypes.fields import GenericForeignKey, GenericRelation
from django.contrib.contenttypes.models import ContentType
from django.db.models.signals import post_delete
from django.dispatch import receiver

class Resource(models.Model):
    content_type = models.ForeignKey(ContentType)
    object_id = models.PositiveIntegerField()
    content_object = GenericForeignKey()

class Account(models.Model):
    name = models.CharField(max_length=10)
    resources = GenericRelation(Resource)

class Order(models.Model):
    account = models.ForeignKey(Account)
    content_type = models.ForeignKey(ContentType)
    object_id = models.PositiveIntegerField()
    content_object = GenericForeignKey()

class MetricStorage(models.Model):
    order = models.ForeignKey(Order)

@receiver(post_delete, dispatch_uid="orders.cancel_orders")
def cancel_orders(sender, **kwargs):
    instance = kwargs['instance']
    print('delete', sender, instance, instance.pk)
    if sender == Resource:
        related = instance.content_object
        if related:
            ct = ContentType.objects.get_for_model(related)
            order = Order.objects.get(content_type=ct, object_id=related.pk)
            order.metricstorage_set.create()
            order.save()
        else:
            print('related is None')

And this test code

from test.models import Account, Resource, Order

account = Account.objects.create(name='John')
resource = Resource.objects.create(content_object=account)
order = Order.objects.create(account=account, content_object=account)
account.delete()

In order to properly test this you should make tries restarting the interpreter a handful of times, and then you'll see two different results.

This that I consider correct:

delete <class 'test.models.Order'> Order object 384
delete <class 'test.models.Account'> Account object 124
delete <class 'test.models.Resource'> Resource object 307
related is None

And the IntegrityError which I consider to be a bug

delete <class 'test.models.Resource'> Resource object 311
delete <class 'test.models.Order'> Order object 388
delete <class 'test.models.Account'> Account object 128
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/base.py", line 872, in delete
    collector.delete()
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/deletion.py", line 314, in delete
    sender=model, instance=obj, using=self.using
  File "/usr/local/lib/python3.4/dist-packages/django/db/transaction.py", line 232, in __exit__
    connection.commit()
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 173, in commit
    self._commit()
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 142, in _commit
    return self.connection.commit()
  File "/usr/local/lib/python3.4/dist-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python3.4/dist-packages/django/utils/six.py", line 658, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 142, in _commit
    return self.connection.commit()
django.db.utils.IntegrityError: insert or update on table "test_metricstorage" violates foreign key constraint "test_metricstorage_order_id_730c757c66b8c627_fk_test_order_id"
DETAIL:  Key (order_id)=(388) is not present in table "test_order".

Notice how the order in which related objects are deleted is different, I believe this to be the source of the problem.

I've noticed this problem on 1.7 and now 1.8, not tested with master.

Change History (5)

comment:1 by Tim Graham, 9 years ago

Description: modified (diff)
Triage Stage: UnreviewedAccepted

I could also reproduce on master with your example code.

comment:2 by sww, 9 years ago

Looks like this behavior's been here a while - looks like commit 272de9eb6baad45abec029aae92c2b7d9478c841 introduced the bug, so it's been a bug since 1.6.

comment:3 by Laura Feier, 8 years ago

Cc: feierlaura10@… added
Has patch: set

PR

Tests pass under SQLite, MySQL and PostgreSQL

comment:4 by Simon Charette, 8 years ago

Triage Stage: AcceptedReady for checkin

comment:5 by Tim Graham <timograham@…>, 8 years ago

Resolution: fixed
Status: newclosed

In 7862cbda:

Fixed #24576 -- Made deletion of related objects deterministic.

Note: See TracTickets for help on using tickets.
Back to Top