Opened 6 months ago

Closed 5 months ago

Last modified 5 months ago

#30325 closed Bug (fixed)

Inconsistent count() behaviour on reverse relations in ManyToManyFields with custom model manager

Reported by: Tobias Kunze Owned by: Tobias Kunze
Component: Database layer (models, ORM) Version: 2.2
Severity: Release blocker Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When looking up a reverse relation on a ManyToManyField that uses a custom model manager, not using all() before calling count() will not use the custom model manager. This results in obj.m2m_lookup.count() yielding a different result than obj.m2m_lookup.all().count, which is new behaviour in 2.2, and unexpected (at least to me).

I've written up a minimal test case to showcase the issue. Inlining here:

from django.db import models


class Author(models.Model):
    name = models.CharField(max_length=100)


class BookManager(models.Manager):
    def get_queryset(self):
        return super().get_queryset().exclude(state='deleted')


class Book(models.Model):
    title = models.CharField(max_length=100)
    authors = models.ManyToManyField(to=Author, related_name='books')
    state = models.CharField(choices=(('regular', 'regular'), ('deleted', 'deleted')), default='regular', max_length=100)

    objects = BookManager()

a = Author.objects.create(name='Bill')
b1 = Book.objects.create(title='Sonnets', state='deleted')
b2 = Book.objects.create(title='Hamlet')
b1.authors.add(a)
b2.authors.add(a)
a.books.count()  # Returns 2
a.books.all().count()  # Returns 1

Change History (12)

comment:1 Changed 6 months ago by Pizzolato Davide

Resolution: invalid
Status: newclosed

a.books.all() is managed by the BookManager so it exclude the 'deleted' book b1

comment:2 Changed 6 months ago by felixxm

Resolution: invalid
Severity: NormalRelease blocker
Status: closednew
Triage Stage: UnreviewedAccepted

Reproduced at 1ffddfc233e2d5139cc6ec31a4ec6ef70b10f87f and works in Django 2.1.

Thanks for the report. Can you bisect to check which commit change this behavior?

comment:3 Changed 6 months ago by Pizzolato Davide

Resolution: invalid
Status: newclosed

This is not a bug!

comment:4 Changed 6 months ago by felixxm

Resolution: invalid
Status: closednew

PizzolatoDavide Please do not shout.

a.books.count() != a.books.all().count() for me it is a bug, moreover this behavior has changed in Django 2.2.

comment:5 Changed 6 months ago by Tobias Kunze

This is not a bug!

Could you elaborate on this? As it is changed behaviour from Django 2.1 and previous versions, it's either a bug/regression, or a documentation bug by not mentioning this changed behaviour in the release notes, in my opinion. Up until now I've assumed that something.related.count() and something.related.all().count() will yield the same result, and if this is not true, it should be documented. (Or is it documented? Could you point me at the right place to look?)

comment:6 Changed 5 months ago by Marten Kenbeek

Regression caused by 1299421 (#29725). .count() and .exists() on a many-to-many relation now use the through model's base manager instead of the related model's default manager.

comment:7 Changed 5 months ago by Tobias Kunze

As per discussion at DjangoCon Europe, the patch in question will probably be reverted, closing this issue (and re-opening #29725, potentially). I've submitted regression tests: https://github.com/django/django/pull/11205

comment:8 Changed 5 months ago by felixxm

Has patch: set
Owner: changed from nobody to Tobias Kunze
Status: newassigned
Triage Stage: AcceptedReady for checkin

comment:9 Changed 5 months ago by Mariusz Felisiak <felisiak.mariusz@…>

Resolution: fixed
Status: assignedclosed

In 5f7991c4:

Fixed #30325 -- Reverted "Fixed #29725 -- Removed unnecessary join in QuerySet.count() and exists() on a many-to-many relation."

This reverts commit 1299421cadc4fcf63585f2f88337078e43e660e0 due to
a regression with custom managers.

comment:10 Changed 5 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In 9ac8520:

Refs #30325 -- Added tests for using count()/exists() with custom managers and reverse M2M relations.

comment:11 Changed 5 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In e8de1cc9:

[2.2.x] Fixed #30325 -- Reverted "Fixed #29725 -- Removed unnecessary join in QuerySet.count() and exists() on a many-to-many relation."

This reverts commit 1299421cadc4fcf63585f2f88337078e43e660e0 due to
a regression with custom managers.

Backport of 5f7991c42cff73b6278106d499d719b726f85ead from master

comment:12 Changed 5 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In 5289d4f:

[2.2.x] Refs #30325 -- Added tests for using count()/exists() with custom managers and reverse M2M relations.

Backport of 9ac8520fcde29840a1345be19d80dbda53aa6d03 from master

Note: See TracTickets for help on using tickets.
Back to Top