id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
26522	Non-deterministic crash in django.db.models.sql.Query.combine()	Ole Laursen	nobody	"Hi,

There's a logical error somewhere in Query.combine or friends.

I have a pretty complicated query with many-to-many self-joins and ORs, and after the upgrade to Django 1.9 and Python 3 combine() sometimes, but not always, crashes with the assertion error
{{{
assert set(change_map.keys()).intersection(set(change_map.values())) == set()
}}}

Inspecting the change_map, it does indeed contain a circular reference (BTW shouldn't you use ""not"" to test for an empty set rather than comparing with set()?).

At first, I didn't understand how it could crash sometimes and sometimes not - we're talking about the same query being executed through a view. But when I examine change_map, its content does indeed change from reload to reload - I suppose this may have something to do with the order of dicts/sets the combine() algorithm is using.

Here comes some code, I cut out a bunch of unused stuff, but otherwise it's the same:

{{{
class Invoice(models.Model):
    customer = models.ForeignKey(Customer)

    date_created = models.DateField(default=datetime.date.today, db_index=True)
    date_sent = models.DateField(null=True, blank=True)
    date_due = models.DateField(null=True, blank=True)
    date_paid = models.DateField(null=True, blank=True)
    date_credited = models.DateField(null=True, blank=True)
    date_collect = models.DateField(null=True, blank=True)

    invoice_type = models.CharField(default=""invoice"", max_length=32)

    reminders = models.ManyToManyField(""Invoice"", related_name=""reminded_set"", blank=True)
    reminder_counter = models.IntegerField(null=True, blank=True)
}}}

That's the model, and in the view:

{{{
    import datetime
    from django.db.models import Q

    date = datetime.datetime.now()

    invoices = Invoice.objects.filter(
        Q(date_created__lte=date),
        Q(date_paid__gt=date) | Q(date_paid=None),
        Q(date_credited__gt=date) | Q(date_credited=None),
        customer=1,
    )

    filtered_invoices = Invoice.objects.none()

    not_due = Q(date_due__gte=date) | Q(date_due=None)
    not_reminded_yet = ~Q(reminders__date_created__lte=date)
    not_collected = Q(date_collect__gt=date) | Q(date_collect=None)

    filtered_invoices |= invoices.filter(not_due, not_collected, date_sent__lte=date, invoice_type=""invoice"")

    filtered_invoices |= invoices.filter(not_collected, not_reminded_yet, date_sent__lte=date, date_due__lt=date, invoice_type=""invoice"")

    for r in [1, 2, 3]:
        qs = invoices.filter(not_collected, reminders__date_created__lte=date, reminders__reminder_counter=r, invoice_type=""invoice"")
        for i in range(r + 1, 3 + 1):
            qs = qs.filter(~Q(reminders__reminder_counter=i) | Q(reminders__reminder_counter=i, reminders__date_created__gt=date))
        filtered_invoices |= qs
}}}

I realize it's pretty complicated but I'm not sure what's essential and what's not. I hope someone knowledgeable of how combine() works can figure out why ordering in some cases matters."	Bug	closed	Database layer (models, ORM)	1.9	Normal	fixed			Ready for checkin	1	0	0	0	0	0