annotations + base_manager_name + instance.save() raises exception

[James Addison]

With a models.py like:

class ItemManager(models.Manager):
    def get_queryset(self):
        return super().get_queryset().annotate(
            rating_avg=models.Avg('review__rating')
        )


class Item(models.Model):
    name = models.CharField(max_length=10, default='')

    objects = ItemManager()

    class Meta:
        base_manager_name = 'objects'


class Review(models.Model):
    item = models.ForeignKey(Item)
    rating = models.PositiveSmallIntegerField(default=0)

If an Item instance is modified and item.save() is called like this:

    item = Item.objects.first()
    item.name = 'abc'
    item.save()

It will raise this exception:

Traceback (most recent call last):
  File "/home/myuser/django_annotations_bug/the_bug/tests.py", line 9, in test_bug_will_be_triggered
    item.save()
  File "/home/myuser/.virtualenvs/django_annotations_bug/lib/python3.6/site-packages/django/db/models/base.py", line 808, in save
    force_update=force_update, update_fields=update_fields)
  File "/home/myuser/.virtualenvs/django_annotations_bug/lib/python3.6/site-packages/django/db/models/base.py", line 838, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/home/myuser/.virtualenvs/django_annotations_bug/lib/python3.6/site-packages/django/db/models/base.py", line 905, in _save_table
    forced_update)
  File "/home/myuser/.virtualenvs/django_annotations_bug/lib/python3.6/site-packages/django/db/models/base.py", line 955, in _do_update
    return filtered._update(values) > 0
  File "/home/myuser/.virtualenvs/django_annotations_bug/lib/python3.6/site-packages/django/db/models/query.py", line 664, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
  File "/home/myuser/.virtualenvs/django_annotations_bug/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1199, in execute_sql
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "/home/myuser/.virtualenvs/django_annotations_bug/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 894, in execute_sql
    raise original_exception
  File "/home/myuser/.virtualenvs/django_annotations_bug/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 884, in execute_sql
    cursor.execute(sql, params)
  File "/home/myuser/.virtualenvs/django_annotations_bug/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/home/myuser/.virtualenvs/django_annotations_bug/lib/python3.6/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/myuser/.virtualenvs/django_annotations_bug/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/home/myuser/.virtualenvs/django_annotations_bug/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/home/myuser/.virtualenvs/django_annotations_bug/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 328, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: sub-select returns 2 columns - expected 1

I have attached a simple sample project that reproduces this error. To set it up:

tar xzvf bug.tgz
cd django_annotations_bug
mkvirtualenv -ppython3 buggy_mcbugface
pip install -r requirements.txt
./manage.py test

Related to #19513, #28374, #26539 - this those are related to .update() calls, mostly. This example does not use the annotation in the save/updating of the item, so is different I think.

[James Addison]

sample project to reproduce bug

[James Addison]

Adding this hack to the Item model addresses the issue for now. Yes, I realize this is overriding an undocumented, internal Django Model method and is thus fragile to future changes - but it's a hack that lets me proceed for now and hopefully helps the Django team.

class Item(models.Model):
    ....
    def _do_update(self, base_qs, *args, **kwargs):
        qs = base_qs.all()
        qs.query.annotations.clear()
        return super()._do_update(qs, *args, **kwargs)

[Tim Graham]

It looks like this was also reported on the GitHub page of a84344bc539c66589c8d4fe30c6ceaecf8ba1af3.

[shangdahao]

​PR

[Carlton Gibson]

Comments on ​PR

[Tim Graham <timograham@…>]

In 8dc675d:

Fixed #28838 -- Fixed Model.save() crash if the base manager annotates with a related field.