Inconsistent count() behaviour on reverse relations in ManyToManyFields with custom model manager

[Tobias Kunze]

When looking up a reverse relation on a ManyToManyField that uses a custom model manager, not using all() before calling count() will not use the custom model manager. This results in obj.m2m_lookup.count() yielding a different result than obj.m2m_lookup.all().count, which is new behaviour in 2.2, and unexpected (at least to me).

I've written up a ​minimal test case to showcase the issue. Inlining here:

from django.db import models


class Author(models.Model):
    name = models.CharField(max_length=100)


class BookManager(models.Manager):
    def get_queryset(self):
        return super().get_queryset().exclude(state='deleted')


class Book(models.Model):
    title = models.CharField(max_length=100)
    authors = models.ManyToManyField(to=Author, related_name='books')
    state = models.CharField(choices=(('regular', 'regular'), ('deleted', 'deleted')), default='regular', max_length=100)

    objects = BookManager()

a = Author.objects.create(name='Bill')
b1 = Book.objects.create(title='Sonnets', state='deleted')
b2 = Book.objects.create(title='Hamlet')
b1.authors.add(a)
b2.authors.add(a)
a.books.count()  # Returns 2
a.books.all().count()  # Returns 1

[Pizzolato Davide]

a.books.all() is managed by the BookManager so it exclude the 'deleted' book b1

[Mariusz Felisiak]

Reproduced at 1ffddfc233e2d5139cc6ec31a4ec6ef70b10f87f and works in Django 2.1.

Thanks for the report. Can you bisect to check which commit change this behavior?

[Pizzolato Davide]

This is not a bug!

[Mariusz Felisiak]

PizzolatoDavide Please do not shout.

a.books.count() != a.books.all().count() for me it is a bug, moreover this behavior has changed in Django 2.2.

[Tobias Kunze]

This is not a bug!

Could you elaborate on this? As it is changed behaviour from Django 2.1 and previous versions, it's either a bug/regression, or a documentation bug by not mentioning this changed behaviour in the release notes, in my opinion. Up until now I've assumed that something.related.count() and something.related.all().count() will yield the same result, and if this is not true, it should be documented. (Or is it documented? Could you point me at the right place to look?)

[Marten Kenbeek]

Regression caused by 1299421 (#29725). .count() and .exists() on a many-to-many relation now use the through model's base manager instead of the related model's default manager.

[Tobias Kunze]

As per discussion at DjangoCon Europe, the patch in question will probably be reverted, closing this issue (and re-opening #29725, potentially). I've submitted regression tests: ​https://github.com/django/django/pull/11205

[Mariusz Felisiak]

​PR

[Mariusz Felisiak <felisiak.mariusz@…>]

In 5f7991c4:

Fixed #30325 -- Reverted "Fixed #29725 -- Removed unnecessary join in QuerySet.count() and exists() on a many-to-many relation."

This reverts commit 1299421cadc4fcf63585f2f88337078e43e660e0 due to
a regression with custom managers.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 9ac8520:

Refs #30325 -- Added tests for using count()/exists() with custom managers and reverse M2M relations.

[Mariusz Felisiak <felisiak.mariusz@…>]

In e8de1cc9:

[2.2.x] Fixed #30325 -- Reverted "Fixed #29725 -- Removed unnecessary join in QuerySet.count() and exists() on a many-to-many relation."

This reverts commit 1299421cadc4fcf63585f2f88337078e43e660e0 due to
a regression with custom managers.

Backport of 5f7991c42cff73b6278106d499d719b726f85ead from master

[Mariusz Felisiak <felisiak.mariusz@…>]

In 5289d4f:

[2.2.x] Refs #30325 -- Added tests for using count()/exists() with custom managers and reverse M2M relations.

Backport of 9ac8520fcde29840a1345be19d80dbda53aa6d03 from master