Table for Model that has been proxied may not be created

[Marc Fargas]

Hi there,

Imagine you have

app1/models.py::

    from app2.models import NiceModel
    class ProxyModel(NiceModel):
        class Meta:
            proxy = True

app2/models.py::

   from django.db import models

   class NiceModel(models.Model):
       pass

With this code, "manage.py syncdb" will likely first "see" 'ProxyModel'. In [source:django/trunk/django/core/management/commands/syncdb.py#11780 syncdb.py]:

• It will 'try' to (line 70) create a table for it, without doing anything.
• But adding "app2_nicemodel" (line 82) to the list of tables "it has taken care of"
• For that, when syncdb passes throught NiceModel it won't create any table as 'app2_nicemodel' (line 66) appears as created.

To reproduce that you need syncdb to be taking into account the "proxy" models before the realones (may happen when the "proxying" app is alphabetically before the real one).

[flyingfred0]

This bug appears to have been fixed by [12597]. The attached patch is simply an additional test that reproduces the issue prior to [12597].

[flyingfred0]

Patch to fix 1.1.X branch

[flyingfred0]

The proxy_model_inheritance_test patch is valid for both trunk and the 1.1.X branch. The syncdb_proxy_model patch fixes this issue for the 1.1.X branch.

[Karen Tracey]

(In [12829]) [1.1.X] Fixed #12286: Ensure proxied model's table is created.
Thanks to telenieko for the report and flyingfred0 for test and fix.

Test is r12828 from trunk.

[Tom X. Tobin]

I'm seeing the new proxy_model_inheritance test added in r12828 raise an exception under PostgreSQL (backend postgresql_psycopg2):

======================================================================
ERROR: test_table_exists (modeltests.proxy_model_inheritance.tests.ProxyModelInheritanceTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ttobin/repo/django/tests/modeltests/proxy_model_inheritance/tests.py", line 35, in test_table_exists
    self.assertEquals(NiceModel.objects.all().count(), 0)
  File "/Users/ttobin/lib/python2/packages/django/db/models/query.py", line 324, in count
    return self.query.get_count(using=self.db)
  File "/Users/ttobin/lib/python2/packages/django/db/models/sql/query.py", line 368, in get_count
    number = obj.get_aggregation(using=using)[None]
  File "/Users/ttobin/lib/python2/packages/django/db/models/sql/query.py", line 340, in get_aggregation
    result = query.get_compiler(using).execute_sql(SINGLE)
  File "/Users/ttobin/lib/python2/packages/django/db/models/sql/compiler.py", line 729, in execute_sql
    cursor.execute(sql, params)
  File "/Users/ttobin/lib/python2/packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
    return self.cursor.execute(query, args)
DatabaseError: relation "app2_nicemodel" does not exist
LINE 1: SELECT COUNT(*) FROM "app2_nicemodel"

[Karen Tracey]

Odd. Test passes when run with sqlite, MySQL (both InnoDB and MyISAM), and Oracle. Fails with PostgreSQL, on both trunk and the 1.1.X branch. I've no idea why.

[Russell Keith-Magee]

(In [12846]) Fixed #12286 -- Modified the test case added in r12828 so that it passes under Postgres.

[Ramiro Morales]

While debugging another unrelated issue I found that the situation described by the OP is still happening:

When the 'app1' app (the one containing the proxy model) from the 'proxy_model_inheritance' tests added to fix this ticket is processed in this ​loop, the "app2_nicemodel" table corresponding to the 'app2.models.NiceModel' is being reported as part of this app's tables manifest.

Later, when the 'app2' application is processed the same table is reported again and blindly added again to the 'tables' collection (that is implemented with a list).

I'm reopening because this is a code correctness fix, I'm dropping the 1.2 milestone though.

[Joost Cassee]

We encounter this problem during syncdb (and by extension test) of our project using Django trunk (r12970); the project has become unusable with Postgresql. The latest patch by ramiro (12286-r12948.diff) does not fix it for us. If there is anything we can do to help debugging the problem, please let up know.

[Russell Keith-Magee]

If there is a still a regressing behavior, that needs to be addressed before 1.2

[Ramiro Morales]

Replying to jcassee:

We encounter this problem during syncdb (and by extension test) of our project using Django trunk (r12970); the project has become unusable with Postgresql. The latest patch by ramiro (12286-r12948.diff) does not fix it for us. If there is anything we can do to help debugging the problem, please let up know.

Isolation of a reduced test case that shows the problem you are experiencing and posting it here would be very helpful.

[Joost Cassee]

Replying to ramiro:

Isolation of a reduced test case that shows the problem you are experiencing and posting it here would be very helpful.

I would, but the test runner calls syncdb and the exception is thrown. So it would have to be somthing with a reduced INSTALLED_APPLICATIONS list. I may try the above snippets, but I have a suspicion that it will pass the test. I will try anyway and report back.

[Joost Cassee]

Running the proxy_model_inheritance test passes after cleaning out the INSTALLED_APPS list. Isolating a reduced test case will take some time, as the package that contains the proxy models depends on a number of apps.

[Russell Keith-Magee]

Unfortunately, unless someone can provide a test case to verify that this problem still exists, we'll need to drop this ticket from the release blocker list for 1.2. If you can provide *any* solid evidence that this problem still exists, we need to see it very soon.

[harm]

We see this problem too. I can confirm its not fixed.

I've however not (yet) been able to reproduce the issue with a cleaned out INSTALLED_APPS. When trying then all just seems to work. The ordering of of table creation seems to influence this bug.

[harm]

here is a testcase:

settings.py

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.messages',

    'tagging',
    'anapp',

)

where tagging is django-tagging. (​http://code.google.com/p/django-tagging/)

an anapp is an almost empty app with with the following models

from django.db import models

from tagging.models import Tag, TaggedItem

class TagProxy(Tag):
    class Meta:
        proxy = True

class TaggedItemProxy(TaggedItem):
    class Meta:
        proxy = True

Now when doing a manage.py syncdb or manage.py test the Tag & TaggedItem tables are _not_ created.

(python2.5)harm@dev2:~/svn/idq/trunk/projects/idqsso/django/projects/ep_auth_demo$ python manage.py test
Creating test database 'default'...
Creating table django_session
Creating table django_site
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_user_permissions
Creating table auth_user_groups
Creating table auth_user
Creating table auth_message
Creating table django_content_type
Installing index for auth.Permission model      <<<<<<<<<<<<<<<< before here the tagging tables should be created

[Russell Keith-Magee]

Possible fix for syncdb problems with proxy models?

[Russell Keith-Magee]

@harm: I can't reproduce - using that example, I see the tagging tables synchronized, both under syncdb and test, exactly where you indicate that they aren't.

However, looking at syncdb, it occurs to me that the problem could be one of dictionary ordering. Can you try the patch I just uploaded to see if it fixes the problem for you?

[harm]

I can no longer reproduce the problem on trunk revision [13027].
(NB: without your patch: t12286.diff ).

(seen on [12403] and a unknown trunk version of around 1.5 weeks ago)

Do others still see the issue ? (jcassee, ramiro)

[Marcus Fredriksson]

[12590] Is the last revision where I can reproduce the problem. Seems like [12597] fixed it.

[Joost Cassee]

Replying to russellm:

However, looking at syncdb, it occurs to me that the problem could be one of dictionary ordering. Can you try the patch I just uploaded to see if it fixes the problem for you?

Yes, your patch fixes the problem for us. Thanks a lot, Russell! (Note that without the patch syncdb still fails, even on r12970.)

[Russell Keith-Magee]

(In [13028]) Fixed #12286 -- Corrected order of table creation in syncdb so that it doesn't cause problems with proxied models. Thanks to everyone that helped to narrow down the problem.

[Jacob]

Milestone 1.2 deleted

[Aymeric Augustin]

In [17712]:

Added cleanup code to the proxy_model_inheritance tests. Refs #12286, #16329.

[Aymeric Augustin]

In case someone wants to improve these tests later -- there's probably a better way -- I'm attaching a patch that re-introduces the original bug and cases these tests to fail.