Django

Code

Ticket #8990 (closed: fixed)

Opened 2 years ago

Last modified 1 year ago

Cannot save objects when using extra() in default manager

Reported by: miracle2k Assigned to: mtredinnick
Milestone: Component: Database layer (models, ORM)
Version: SVN Keywords:
Cc: Triage Stage: Accepted
Has patch: 1 Needs documentation: 0
Needs tests: 0 Patch needs improvement: 0

Description

Possibly related to #5768 and #3358 (FIXME: select_related isn't supported in values() comment in source).

The last test should not throw an exception:

from django.db import models

class Shelf(models.Model):
    name = models.CharField(max_length=50)

    def __unicode__(self):
        return u"%s the shelf" % self.name


class BookManager(models.Manager):
    def get_query_set(self):
        return super(BookManager, self).get_query_set().\
            select_related('shelf').\
            extra(select={'shelf_name': 'extra_save_shelf.name'})


class Book(models.Model):
    name = models.CharField(max_length=50)
    shelf = models.ForeignKey(Shelf)

    objects = BookManager()

    def __unicode__(self):
        return u"%s the book" % self.name


__test__ = {'API_TESTS':"""
>>> shelf = Shelf(name="Foo")
>>> shelf.save()
>>> book = Book(name="bar", shelf=shelf)
>>> book.save()              # initial save works fine

>>> Book.objects.get(pk=1)   # querying works as well
<Book: bar the book>

# subsequent saves fail due to missing select_related
>>> book.save()
Traceback (most recent call last):
    ...
OperationalError: no such column: extra_save_shelf.name
"""
}

The problem seems to be that the .extra().values() query that is used to determine whether to do an update or insert doesn't use select_related(), while the manager's extra clause() relies on the tables from select_related existing.

Is there a reason why save's extra() query would need to consider the manager's at all? Wouldn't overwriting existing extra()s be the right thing to do in any case?

Attachments

reset-extra.diff (7.0 kB) - added by miracle2k on 09/17/08 10:46:49.

Change History

09/17/08 10:46:49 changed by miracle2k

  • attachment reset-extra.diff added.

09/17/08 10:51:14 changed by miracle2k

  • needs_better_patch changed.
  • has_patch set to 1.
  • needs_tests changed.
  • needs_docs changed.

Added a simple patch that allows to reset a query's extra() data by passing False, and uses this for the values() query. It might be nicer to give extra a "clear" parameter instead (you probably don't want to reset the whole extra rather than specific options anyway most of the time), but I wanted to avoid changing the method signature.

09/17/08 18:58:09 changed by mtredinnick

  • stage changed from Unreviewed to Accepted.

This is actually just a symptom of a much larger problem; pretty much all custom managers are inappropriate to use for saving. Ultimately it comes down to using "default manager" for something that isn't really appropriate, but a lot of people are doing that.

It's fairly tricky to fix the root problem properly (since there are some very rare cases like GeoManager? that do need to be involved, possibly), but I'm working on it.

02/25/09 13:51:44 changed by

  • milestone deleted.

Milestone post-1.0 deleted

03/13/09 01:58:28 changed by mtredinnick

  • owner changed from nobody to mtredinnick.
  • status changed from new to assigned.

03/14/09 22:41:33 changed by mtredinnick

  • status changed from assigned to closed.
  • resolution set to fixed.

(In [10056]) Use plain model.Manager, or suitable proxy, for model saving.

We can't use the default manager in Model.save_base(), since we need to retrieve existing objects which might be filtered out by that manager. We now always use a plain Manager instance at that point (or something that can replace it, such as a GeoManager?), making all existing rows in the database visible to the saving code.

The logic for detecting a "suitable replacement" plain base is the same as for related fields: if the use_for_related_fields is set on the manager subclass, we can use it. The general requirement here is that we want a base class that returns the appropriate QuerySet? subclass, but does not restrict the rows returned.

Fixed #8990, #9527.

Refs #2698 (which is not fixed by this change, but it's the first part of a larger change to fix that bug.)

03/14/09 22:45:32 changed by mtredinnick

(In [10059]) [1.0.X] Use plain model.Manager, or suitable proxy, for model saving.

We can't use the default manager in Model.save_base(), since we need to retrieve existing objects which might be filtered out by that manager. We now always use a plain Manager instance at that point (or something that can replace it, such as a GeoManager?), making all existing rows in the database visible to the saving code.

The logic for detecting a "suitable replacement" plain base is the same as for related fields: if the use_for_related_fields is set on the manager subclass, we can use it. The general requirement here is that we want a base class that returns the appropriate QuerySet? subclass, but does not restrict the rows returned.

Fixed #8990, #9527.

Refs #2698 (which is not fixed by this change, but it's the first part of a larger change to fix that bug.)

Backport of r10056 from trunk.


Add/Change #8990 (Cannot save objects when using extra() in default manager)




Change Properties
Action