Django

Code

Ticket #12767 (new)

Opened 6 months ago

Last modified 3 months ago

Problem running syncdb with a multi-db router that restricts auth to a single database

Reported by: russellm Assigned to: nobody
Milestone: Component: Database layer (models, ORM)
Version: SVN Keywords:
Cc: klemens@fetzig.at, kmpm@birchroad.net, jonj@ghs.com, digitalxero@gmail.com Triage Stage: Someday/Maybe
Has patch: 1 Needs documentation: 0
Needs tests: 0 Patch needs improvement: 1

Description

Via Klemens Mantzos on django-users:

If i do

python manage.py syncdb --database=default

before i sync the users database this error comes up (which seems ok to me):

django.db.utils.DatabaseError: (1146, "Table 'multidb_user.auth_permission' doesn't exist")

but if i do this:

python manage.py syncdb --database=users

i get that every time (no matter if django_content_type already exists in the default db or not):

django.db.utils.DatabaseError: (1146, "Table 'multidb_user.django_content_type' doesn't exist")

Router config

# don't know but OtherRouter is maybe obsolete.
DATABASE_ROUTERS = ['dbrouter.AuthRouter', 'dbrouter.OtherRouter']

project/dbrouter.py

class AuthRouter(object):
  """A router to control all database operations on models in
  the contrib.auth application"""

  ....

  def allow_syncdb(self, db, model):
      "Make sure the auth app only appears on the 'credentials' db"
      if db == 'users':
          return model._meta.app_label == 'auth'
      elif model._meta.app_label == 'auth':
          return False
      return None


class OtherRouter(object):
  """A router that sets up a simple master/slave configuration"""

  ...

  def allow_syncdb(self, db, model):
      "Explicitly put all models on all databases."
      return True

Attachments

block_contenttypes.diff (0.8 kB) - added by adamv on 02/09/10 17:59:28.
Hack: blocks command if ContentType? isn't allowed to sync into the target db.

Change History

02/03/10 18:00:38 changed by russellm

  • needs_better_patch changed.
  • stage changed from Unreviewed to Accepted.
  • version changed from 1.2-alpha to SVN.
  • needs_tests changed.
  • needs_docs changed.

The issue appears to be the way that the contenttype post-syncdb handler deals with a multi-db situation.

02/03/10 18:24:06 changed by fetzig

  • cc set to klemens@fetzig.at.

02/03/10 23:51:17 changed by kmpm

  • cc changed from klemens@fetzig.at to klemens@fetzig.at, kmpm@birchroad.net.

02/05/10 12:17:29 changed by jonj

  • cc changed from klemens@fetzig.at, kmpm@birchroad.net to klemens@fetzig.at, kmpm@birchroad.net, jonj@ghs.com.

02/09/10 17:59:28 changed by adamv

  • attachment block_contenttypes.diff added.

Hack: blocks command if ContentType? isn't allowed to sync into the target db.

03/10/10 06:32:59 changed by russellm

After closer inspection, I'm going to mark this someday/maybe.

The underlying problem here is that auth has foreign keys to contenttypes. This means that you need to deal with referential integrity across databases, which is a non-trivial problem to solve (other than the obvious workaround of putting contenttypes on the same database as auth)

An alternative solution would be to modify the contenttypes syncdb handler so that contenttypes can be synchronized onto multiple databases. However, this would require lots of extra logic to ensure that databases remained in sync, including ensuring that primary key allocation for content types is consistent between databases.

However, I will make a change to the multi-db docs -- they currently suggest that routers can be used to put auth onto a separate database, and while this is technically true, it won't be true for many uses in practice. Better to avoid the implication altogether.

03/10/10 06:35:55 changed by russellm

(In [12751]) Refs #12767 -- Modified the multi-db docs to remove the implication that contrib.auth can be easily put on a separate database.

03/10/10 06:38:19 changed by russellm

  • stage changed from Accepted to Someday/Maybe.
  • milestone deleted.

03/11/10 06:30:19 changed by fetzig

maybe its a stupid question but I just don't get it, and need to ask:

@russellm does that mean that its required to keep models with foreign keys to each other together in one db (even with configured router)?

or is it just a "special" problem with the syncdb of contenttypes?

thx for your time russel.

greets, klemens

03/11/10 06:52:04 changed by russellm

@fetzig -- This is not specific to ContentTypes?. Foreign Keys can't cross databases. This is all a consequence of referential integrity -- a database can't guarantee referential integrity if it doesn't have access to the ground truth. There might be ways to relax this restriction, but it isn't trivial.

04/27/10 13:09:09 changed by Digitalxero

  • cc changed from klemens@fetzig.at, kmpm@birchroad.net, jonj@ghs.com to klemens@fetzig.at, kmpm@birchroad.net, jonj@ghs.com, digitalxero@gmail.com.
  • has_patch set to 1.
  • milestone set to 1.2.

This bug affects any multi-db config that has django.contrib.contenttpes installed even if the models being synced do not have a ForeignKey to ContentType?

My examples, I have a DB of gis data for zip codes, counties / states etc. This DB is used by a number of sites so I want it in it's own DB, and nothing else gets synced there but when I do

manage.py syncdb --database=gis

I get

django.db.utils.DatabaseError: (1146, "Table 'gis.django_content_type' doesn't exist")

None of the models in my gis app have relations to ContentTypes? and no app in the rest of the site has relations to the gis app, this is something I ensure with a DB Router. So this error is coming because the ContentTypes? app wants to track all installed applications, but it should not be tracking items it can have no relation with.

This patch does fix the error

04/27/10 21:36:09 changed by russellm

  • needs_better_patch set to 1.
  • milestone deleted.

The patch doesn't fix the error at all; it hides it. It means that you won't have a content type for the objects in the separate database, which has the potential to cause all sorts of other problems.

I also have a suspicion there might be something wrong with your router; If you're getting errors about "gis.django_content_type", that suggests that something in your routing configuration is pushing contenttype to the gis database, which shouldn't be happening.

04/28/10 07:04:54 changed by Digitalxero

I am fairly sure it is because the update_contenttypes method sets using(db) for all db calls (eg ct.save(using=db))

04/28/10 07:13:24 changed by russellm

You may want to update to trunk. The current source code doesn't force the database at all. There aren't any calls to using() or save(using=)

04/28/10 08:49:47 changed by Digitalxero

Thats what I get for testing with beta1 and not upgrading to trunk when I find a bug. Just upgraded to trunk and verified I no longer see my issue even without this patch.


Add/Change #12767 (Problem running syncdb with a multi-db router that restricts auth to a single database)




Change Properties
Action