#33011 closed Bug (needsinfo)

my_model.manyrelation_set.all() return empty value

Reported by: Floréal Cabanettes Owned by: nobody
Component: Database layer (models, ORM) Version: 3.2
Severity: Normal Keywords: model set querying
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I have a problem which occurred since django 3.2 (no problem with django 3.1).

I use django rest framework. On the view, I have a model "Run" and a model "Experiment". On the view, I get experiments with :
my_experiments = my_run.experiment_set.all()

But since django 3.2, this returns an empty queryset. If I add a breakpoint in pycharm just after this line, the queryset is no more empty.

Also, this line works perferctly:
my_experiments = Experiment.objects.filter(run_id=my_run.id)

But I think the first code is more beautiful... so why this problem?

Change History (5)

comment:1 Changed 12 months ago by Tim Graham

Hi, could you please provide minimal models and code to reproduce the issue? Ideally, you could also bisect to find the commit where the behavior changed.

comment:2 Changed 12 months ago by Floréal Cabanettes

Hi,

I found the commit responsible for this bug:

036f160733 ("Refs #20577 -- Deferred filtering of prefetched related querysets by reverse m2o relation.", 2020-10-06)

The line which cause the problem is on the file django/db/models/fields/related_descriptors.py, line 584 (581 at the commit, 584 on tag 3.2.6) :

queryset._defer_next_filter = True

If I comment this line, the bug disappear.

My code is not public (is for my enterprise, not my choice). I try to describe a similar architecture bellow:

I have 2 models, one is "Run", the other "Experiment", related by a OneToMany relationship:

class Run(model.Model):
    label = models.CharField(max_length=100, null=False, blank=False)
    owner = models.ForeignKey(User, null=True, blank=True, on_delete=models.PROTECT, related_name="run_owner")
    

class Experiment(model.Model):
    run = models.ForeignKey(Run, null=False, on_delete=models.PROTECT)
    index1 = models.ForeignKey(Index, null=False, on_delete=models.PROTECT, related_name="index1")
    index2 = models.ForeignKey(Index, null=True, blank=True, on_delete=models.PROTECT, related_name="index2")

I use the django rest framework to call the server through REST API. For one endpoint, I get a run, and I try to get it's experiments, to then make a copy of them:

@action(detail=True, methods=["post"])
def copy(self, request, pk=None):
    try:
        my_run = self.get_queryset().get(pk=pk)
    except ObjectDoesNotExist:
        return Response({"detail": _("Run with id '%s' does not exists") % pk},
                        status=status.HTTP_404_NOT_FOUND)
    my_experiments = my_run.experiment_set.all()
    for my_experiment in my_experiments:
         my_experiment.pk = None
         ...
         my_experiment.save()

and the problem is that my_experiments is empty with django 3.2.6 (and no more empty with the line commented like said before).

I hope this will help to solve the problem...

comment:3 Changed 12 months ago by Carlton Gibson

Hi Floréal — thanks for the further detail.

Can you give some indication of the prefetch usage in the get_queryset() method here please:

my_run = self.get_queryset().get(pk=pk)

Thanks.

comment:4 in reply to:  3 Changed 12 months ago by Floréal Cabanettes

This apply some filters on the data (like filters given in input).

But it's not linked to this problem. I try to replace this line by:

my_run = self.queryset.get(pk=pk)

or by :

my_run = Run.objects.get(pk=pk)

and the problem is still the same.

Replying to Carlton Gibson:

Hi Floréal — thanks for the further detail.

Can you give some indication of the prefetch usage in the get_queryset() method here please:

my_run = self.get_queryset().get(pk=pk)

Thanks.

comment:5 Changed 12 months ago by Carlton Gibson

Resolution: needsinfo
Status: newclosed

Hi Floréal.

I can't see how to reproduce this with the info given. (There must be something else going on in your project.)

Django's model_fields tests use these models:

class Foo(models.Model):
    a = models.CharField(max_length=10)
    d = models.DecimalField(max_digits=5, decimal_places=3)


def get_foo():
    return Foo.objects.get(id=1).pk


class Bar(models.Model):
    b = models.CharField(max_length=10)
    a = models.ForeignKey(Foo, models.CASCADE, default=get_foo, related_name='bars')

This is equivalent to yours here I think (in the relevant respects).

Adding this test passes:

diff --git a/tests/model_fields/test_foreignkey.py b/tests/model_fields/test_foreignkey.py
index 0cd6d62a55..c51b6d978a 100644
--- a/tests/model_fields/test_foreignkey.py
+++ b/tests/model_fields/test_foreignkey.py
@@ -12,6 +12,14 @@ from .models import Bar, FkToChar, Foo, PrimaryKeyCharModel
 
 class ForeignKeyTests(TestCase):
 
+    def test_reverse_relation(self):
+        a = Foo.objects.create(id=1, a='abc', d=Decimal('12.34'))
+        # Bar FK defaults to Foo with id=1
+        b = Bar.objects.create(b='bcd')
+        c = Bar.objects.create(b='bcd')
+        d = Bar.objects.create(b='bcd')
+        self.assertSequenceEqual(a.bars.all(), [b, c, d])
+
     def test_callable_default(self):
         """A lazy callable may be used for ForeignKey.default."""
         a = Foo.objects.create(id=1, a='abc', d=Decimal('12.34'))

This is already covered elsewhere in the test suite (many times).

Can I ask you to provide a minimal reproduce of your issue?
Thanks.

Note: See TracTickets for help on using tickets.
Back to Top