Problem with ManyToManyFields and syncdb

[Ben Slavin <benjamin.slavin@…>]

The creation of ManyToMany tables in the database through syncdb is prone to failure, and depends on the ordering of INSTALLED_APPS in settings.py.

manage.py syncdb uses a deferral system (seen_models and pending_references) to ensure proper setup of model tables in the database. This allows the tables to be created without references, and constraints to be added after the related tables exist.

Currently, the creation of ManyToMany tables (in django.core.management._get_many_to_many_sql_for_model) does not use a deferral mechanism, and will fail to create the tables properly. If the models are loaded in the right order, this isn't an issue; however, if the they are loaded in reverse order and the database is strict about enforcing references (PostgreSQL), this is problematic.

Note: This doesn't impact MySQL (MyISAM) or sqlite because of the way they handle references -- they don't.

Here's an example:

bar/models.py

from django.db import models
from myproj.foo.models import Foo
class Bar(models.Model):
    foos = models.ManyToManyField(Foo)

foo/models.py

from django.db import models
class Foo(models.Model):
    a = models.IntegerField()

settings.py excerpt (working version)

INSTALLED_APPS = ('m2m.foo','m2m.bar')

settings.py excerpt (non-working version)

INSTALLED_APPS = ('m2m.bar','m2m.foo')

manage.py syncdb output for non-working version

Creating table bar_bar
Traceback (most recent call last):
  File "./manage.py", line 11, in ?
    execute_manager(settings)
  File "/usr/lib/python2.4/site-packages/django/core/management.py", line 1666, in execute_manager
    execute_from_command_line(action_mapping, argv)
  File "/usr/lib/python2.4/site-packages/django/core/management.py", line 1565, in execute_from_command_line
    action_mapping[action](int(options.verbosity), options.interactive)
  File "/usr/lib/python2.4/site-packages/django/core/management.py", line 543, in syncdb
    cursor.execute(statement)
  File "/usr/lib/python2.4/site-packages/django/db/backends/util.py", line 12, in execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python2.4/site-packages/django/db/backends/postgresql/base.py", line 43, in execute
    return self.cursor.execute(sql, [smart_basestring(p, self.charset) for p in params])
psycopg.ProgrammingError: ERROR:  relation "foo_foo" does not exist

CREATE TABLE "bar_bar_foos" (
    "id" serial NOT NULL PRIMARY KEY,
    "bar_id" integer NOT NULL REFERENCES "bar_bar" ("id") DEFERRABLE INITIALLY DEFERRED,
    "foo_id" integer NOT NULL REFERENCES "foo_foo" ("id") DEFERRABLE INITIALLY DEFERRED,
    UNIQUE ("bar_id", "foo_id")
);

The problem is that "foo_id" should be "integer NOT NULL" and an ALTER TABLE statement should be issued for bar_bar_foos after the foo_foo table is created, rather than trying to create the reference initially.

[Russell Keith-Magee]

(In [4780]) Fixed #3779 -- Resolved problem with order of creation of m2m tables during syncdb. Well spotted, Ben Slavin, and thanks for the very helpful test case.

[Ben Slavin <benjamin.slavin@…>]

Fix confirmed.

Thanks Russ