Code

Opened 4 years ago

Closed 19 months ago

#12767 closed Bug (duplicate)

Problem running syncdb with a multi-db router that restricts auth to a single database

Reported by: russellm Owned by: nobody
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords:
Cc: klemens@…, kmpm@…, jonj@…, digitalxero@…, mmitar@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

Via Klemens Mantzos on django-users:

If i do

python manage.py syncdb --database=default

before i sync the users database this error comes up (which seems ok to me):

django.db.utils.DatabaseError: (1146, "Table 'multidb_user.auth_permission' doesn't exist")

but if i do this:

python manage.py syncdb --database=users

i get that every time (no matter if django_content_type already exists in the default db or not):

django.db.utils.DatabaseError: (1146, "Table 'multidb_user.django_content_type' doesn't exist")

Router config

# don't know but OtherRouter is maybe obsolete.
DATABASE_ROUTERS = ['dbrouter.AuthRouter', 'dbrouter.OtherRouter']

project/dbrouter.py

class AuthRouter(object):
  """A router to control all database operations on models in
  the contrib.auth application"""

  ....

  def allow_syncdb(self, db, model):
      "Make sure the auth app only appears on the 'credentials' db"
      if db == 'users':
          return model._meta.app_label == 'auth'
      elif model._meta.app_label == 'auth':
          return False
      return None


class OtherRouter(object):
  """A router that sets up a simple master/slave configuration"""

  ...

  def allow_syncdb(self, db, model):
      "Explicitly put all models on all databases."
      return True

Attachments (1)

block_contenttypes.diff (860 bytes) - added by adamv 4 years ago.
Hack: blocks command if ContentType isn't allowed to sync into the target db.

Download all attachments as: .zip

Change History (19)

comment:1 Changed 4 years ago by russellm

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted
  • Version changed from 1.2-alpha to SVN

The issue appears to be the way that the contenttype post-syncdb handler deals with a multi-db situation.

comment:2 Changed 4 years ago by fetzig

  • Cc klemens@… added

comment:3 Changed 4 years ago by kmpm

  • Cc kmpm@… added

comment:4 Changed 4 years ago by jonj

  • Cc jonj@… added

Changed 4 years ago by adamv

Hack: blocks command if ContentType isn't allowed to sync into the target db.

comment:5 Changed 4 years ago by russellm

After closer inspection, I'm going to mark this someday/maybe.

The underlying problem here is that auth has foreign keys to contenttypes. This means that you need to deal with referential integrity across databases, which is a non-trivial problem to solve (other than the obvious workaround of putting contenttypes on the same database as auth)

An alternative solution would be to modify the contenttypes syncdb handler so that contenttypes can be synchronized onto multiple databases. However, this would require lots of extra logic to ensure that databases remained in sync, including ensuring that primary key allocation for content types is consistent between databases.

However, I will make a change to the multi-db docs -- they currently suggest that routers can be used to put auth onto a separate database, and while this is technically true, it won't be true for many uses in practice. Better to avoid the implication altogether.

comment:6 Changed 4 years ago by russellm

(In [12751]) Refs #12767 -- Modified the multi-db docs to remove the implication that contrib.auth can be easily put on a separate database.

comment:7 Changed 4 years ago by russellm

  • milestone 1.2 deleted
  • Triage Stage changed from Accepted to Someday/Maybe

comment:8 Changed 4 years ago by fetzig

maybe its a stupid question but I just don't get it, and need to ask:

@russellm does that mean that its required to keep models with foreign keys to each other together in one db (even with configured router)?

or is it just a "special" problem with the syncdb of contenttypes?

thx for your time russel.

greets,
klemens

comment:9 Changed 4 years ago by russellm

@fetzig -- This is not specific to ContentTypes. Foreign Keys can't cross databases. This is all a consequence of referential integrity -- a database can't guarantee referential integrity if it doesn't have access to the ground truth. There might be ways to relax this restriction, but it isn't trivial.

comment:10 Changed 4 years ago by Digitalxero

  • Cc digitalxero@… added
  • Has patch set
  • milestone set to 1.2

This bug affects any multi-db config that has django.contrib.contenttpes installed even if the models being synced do not have a ForeignKey to ContentType

My examples, I have a DB of gis data for zip codes, counties / states etc. This DB is used by a number of sites so I want it in it's own DB, and nothing else gets synced there but when I do

manage.py syncdb --database=gis

I get

django.db.utils.DatabaseError: (1146, "Table 'gis.django_content_type' doesn't exist")

None of the models in my gis app have relations to ContentTypes and no app in the rest of the site has relations to the gis app, this is something I ensure with a DB Router. So this error is coming because the ContentTypes app wants to track all installed applications, but it should not be tracking items it can have no relation with.

This patch does fix the error

comment:11 Changed 4 years ago by russellm

  • milestone 1.2 deleted
  • Patch needs improvement set

The patch doesn't fix the error at all; it hides it. It means that you won't have a content type for the objects in the separate database, which has the potential to cause all sorts of other problems.

I also have a suspicion there might be something wrong with your router; If you're getting errors about "gis.django_content_type", that suggests that something in your routing configuration is pushing contenttype to the gis database, which shouldn't be happening.

comment:12 Changed 4 years ago by Digitalxero

I am fairly sure it is because the update_contenttypes method sets using(db) for all db calls (eg ct.save(using=db))

comment:13 Changed 4 years ago by russellm

You may want to update to trunk. The [source:/django/trunk/django/contrib/contenttypes/management.py current source code] doesn't force the database at all. There aren't any calls to using() or save(using=)

comment:14 Changed 4 years ago by Digitalxero

Thats what I get for testing with beta1 and not upgrading to trunk when I find a bug. Just upgraded to trunk and verified I no longer see my issue even without this patch.

comment:15 Changed 4 years ago by mpaolini

maybe related to #14662

comment:16 Changed 3 years ago by julien

  • Severity set to Normal
  • Type set to Bug

comment:17 Changed 2 years ago by mitar

  • Cc mmitar@… added
  • Easy pickings unset
  • UI/UX unset

comment:18 Changed 19 months ago by aaugustin

  • Resolution set to duplicate
  • Status changed from new to closed
  • Triage Stage changed from Someday/Maybe to Accepted

This is a duplicate of #16039 that's fixed in 1.5.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.