Opened 7 years ago

Closed 7 years ago

Last modified 5 years ago

#12286 closed Uncategorized (fixed)

Table for Model that has been proxied may not be created

Reported by: Marc Fargas Owned by: nobody
Component: Core (Management commands) Version: 1.1
Severity: Normal Keywords:
Cc: rvdrijst Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

Hi there,

Imagine you have

app1/models.py::

    from app2.models import NiceModel
    class ProxyModel(NiceModel):
        class Meta:
            proxy = True

app2/models.py::

   from django.db import models

   class NiceModel(models.Model):
       pass

With this code, "manage.py syncdb" will likely first "see" 'ProxyModel'. In [source:django/trunk/django/core/management/commands/syncdb.py#11780 syncdb.py]:

  • It will 'try' to (line 70) create a table for it, without doing anything.
  • But adding "app2_nicemodel" (line 82) to the list of tables "it has taken care of"
  • For that, when syncdb passes throught NiceModel it won't create any table as 'app2_nicemodel' (line 66) appears as created.

To reproduce that you need syncdb to be taking into account the "proxy" models before the realones (may happen when the "proxying" app is alphabetically before the real one).

Attachments (5)

proxy_model_inheritance_test.diff (2.7 KB) - added by flyingfred0 7 years ago.
syncdb_proxy_model.diff (791 bytes) - added by flyingfred0 7 years ago.
Patch to fix 1.1.X branch
12286-r12948.diff (681 bytes) - added by Ramiro Morales 7 years ago.
t12286.diff (1.5 KB) - added by Russell Keith-Magee 7 years ago.
Possible fix for syncdb problems with proxy models?
16329-apply-this-to-reproduce-proxy-model-inheritance-failure.diff (2.2 KB) - added by Aymeric Augustin 5 years ago.

Download all attachments as: .zip

Change History (31)

comment:1 Changed 7 years ago by Russell Keith-Magee

milestone: 1.2
Triage Stage: UnreviewedAccepted

comment:2 Changed 7 years ago by flyingfred0

Owner: changed from nobody to flyingfred0
Status: newassigned

Changed 7 years ago by flyingfred0

comment:3 Changed 7 years ago by flyingfred0

Has patch: set

This bug appears to have been fixed by [12597]. The attached patch is simply an additional test that reproduces the issue prior to [12597].

Changed 7 years ago by flyingfred0

Attachment: syncdb_proxy_model.diff added

Patch to fix 1.1.X branch

comment:4 Changed 7 years ago by flyingfred0

Owner: changed from flyingfred0 to nobody
Status: assignednew

The proxy_model_inheritance_test patch is valid for both trunk and the 1.1.X branch. The syncdb_proxy_model patch fixes this issue for the 1.1.X branch.

comment:5 Changed 7 years ago by Karen Tracey

Resolution: fixed
Status: newclosed

(In [12829]) [1.1.X] Fixed #12286: Ensure proxied model's table is created.
Thanks to telenieko for the report and flyingfred0 for test and fix.

Test is r12828 from trunk.

comment:6 Changed 7 years ago by Tom X. Tobin

Resolution: fixed
Status: closedreopened

I'm seeing the new proxy_model_inheritance test added in r12828 raise an exception under PostgreSQL (backend postgresql_psycopg2):

======================================================================
ERROR: test_table_exists (modeltests.proxy_model_inheritance.tests.ProxyModelInheritanceTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ttobin/repo/django/tests/modeltests/proxy_model_inheritance/tests.py", line 35, in test_table_exists
    self.assertEquals(NiceModel.objects.all().count(), 0)
  File "/Users/ttobin/lib/python2/packages/django/db/models/query.py", line 324, in count
    return self.query.get_count(using=self.db)
  File "/Users/ttobin/lib/python2/packages/django/db/models/sql/query.py", line 368, in get_count
    number = obj.get_aggregation(using=using)[None]
  File "/Users/ttobin/lib/python2/packages/django/db/models/sql/query.py", line 340, in get_aggregation
    result = query.get_compiler(using).execute_sql(SINGLE)
  File "/Users/ttobin/lib/python2/packages/django/db/models/sql/compiler.py", line 729, in execute_sql
    cursor.execute(sql, params)
  File "/Users/ttobin/lib/python2/packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
    return self.cursor.execute(query, args)
DatabaseError: relation "app2_nicemodel" does not exist
LINE 1: SELECT COUNT(*) FROM "app2_nicemodel"

comment:7 Changed 7 years ago by Karen Tracey

Odd. Test passes when run with sqlite, MySQL (both InnoDB and MyISAM), and Oracle. Fails with PostgreSQL, on both trunk and the 1.1.X branch. I've no idea why.

comment:8 Changed 7 years ago by Russell Keith-Magee

Resolution: fixed
Status: reopenedclosed

(In [12846]) Fixed #12286 -- Modified the test case added in r12828 so that it passes under Postgres.

comment:9 Changed 7 years ago by Ramiro Morales

Component: Database layer (models, ORM)django-admin.py
milestone: 1.2
Resolution: fixed
Status: closedreopened

While debugging another unrelated issue I found that the situation described by the OP is still happening:

When the 'app1' app (the one containing the proxy model) from the 'proxy_model_inheritance' tests added to fix this ticket is processed in this loop, the "app2_nicemodel" table corresponding to the 'app2.models.NiceModel' is being reported as part of this app's tables manifest.

Later, when the 'app2' application is processed the same table is reported again and blindly added again to the 'tables' collection (that is implemented with a list).

I'm reopening because this is a code correctness fix, I'm dropping the 1.2 milestone though.

Changed 7 years ago by Ramiro Morales

Attachment: 12286-r12948.diff added

comment:10 Changed 7 years ago by rvdrijst

Cc: rvdrijst added

comment:11 Changed 7 years ago by Joost Cassee

We encounter this problem during syncdb (and by extension test) of our project using Django trunk (r12970); the project has become unusable with Postgresql. The latest patch by ramiro (12286-r12948.diff) does not fix it for us. If there is anything we can do to help debugging the problem, please let up know.

comment:12 Changed 7 years ago by Russell Keith-Magee

milestone: 1.2
Patch needs improvement: set

If there is a still a regressing behavior, that needs to be addressed before 1.2

comment:13 in reply to:  11 ; Changed 7 years ago by Ramiro Morales

Replying to jcassee:

We encounter this problem during syncdb (and by extension test) of our project using Django trunk (r12970); the project has become unusable with Postgresql. The latest patch by ramiro (12286-r12948.diff) does not fix it for us. If there is anything we can do to help debugging the problem, please let up know.

Isolation of a reduced test case that shows the problem you are experiencing and posting it here would be very helpful.

comment:14 in reply to:  13 Changed 7 years ago by Joost Cassee

Replying to ramiro:

Isolation of a reduced test case that shows the problem you are experiencing and posting it here would be very helpful.

I would, but the test runner calls syncdb and the exception is thrown. So it would have to be somthing with a reduced INSTALLED_APPLICATIONS list. I may try the above snippets, but I have a suspicion that it will pass the test. I will try anyway and report back.

comment:15 Changed 7 years ago by Joost Cassee

Running the proxy_model_inheritance test passes after cleaning out the INSTALLED_APPS list. Isolating a reduced test case will take some time, as the package that contains the proxy models depends on a number of apps.

comment:16 Changed 7 years ago by Russell Keith-Magee

Unfortunately, unless someone can provide a test case to verify that this problem still exists, we'll need to drop this ticket from the release blocker list for 1.2. If you can provide *any* solid evidence that this problem still exists, we need to see it very soon.

comment:17 Changed 7 years ago by harm

We see this problem too. I can confirm its not fixed.

I've however not (yet) been able to reproduce the issue with a cleaned out INSTALLED_APPS. When trying then all just seems to work. The ordering of of table creation seems to influence this bug.

comment:18 Changed 7 years ago by harm

here is a testcase:

settings.py

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.messages',

    'tagging',
    'anapp',

)

where tagging is django-tagging. (http://code.google.com/p/django-tagging/)

an anapp is an almost empty app with with the following models

from django.db import models

from tagging.models import Tag, TaggedItem

class TagProxy(Tag):
    class Meta:
        proxy = True

class TaggedItemProxy(TaggedItem):
    class Meta:
        proxy = True

Now when doing a manage.py syncdb or manage.py test the Tag & TaggedItem tables are _not_ created.

(python2.5)harm@dev2:~/svn/idq/trunk/projects/idqsso/django/projects/ep_auth_demo$ python manage.py test
Creating test database 'default'...
Creating table django_session
Creating table django_site
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_user_permissions
Creating table auth_user_groups
Creating table auth_user
Creating table auth_message
Creating table django_content_type
Installing index for auth.Permission model      <<<<<<<<<<<<<<<< before here the tagging tables should be created

Changed 7 years ago by Russell Keith-Magee

Attachment: t12286.diff added

Possible fix for syncdb problems with proxy models?

comment:19 Changed 7 years ago by Russell Keith-Magee

@harm: I can't reproduce - using that example, I see the tagging tables synchronized, both under syncdb and test, exactly where you indicate that they aren't.

However, looking at syncdb, it occurs to me that the problem could be one of dictionary ordering. Can you try the patch I just uploaded to see if it fixes the problem for you?

comment:20 Changed 7 years ago by harm

I can no longer reproduce the problem on trunk revision [13027].
(NB: without your patch: t12286.diff ).

(seen on [12403] and a unknown trunk version of around 1.5 weeks ago)

Do others still see the issue ? (jcassee, ramiro)

comment:21 Changed 7 years ago by Marcus Fredriksson

[12590] Is the last revision where I can reproduce the problem. Seems like [12597] fixed it.

comment:22 in reply to:  19 Changed 7 years ago by Joost Cassee

Replying to russellm:

However, looking at syncdb, it occurs to me that the problem could be one of dictionary ordering. Can you try the patch I just uploaded to see if it fixes the problem for you?

Yes, your patch fixes the problem for us. Thanks a lot, Russell! (Note that without the patch syncdb still fails, even on r12970.)

comment:23 Changed 7 years ago by Russell Keith-Magee

Resolution: fixed
Status: reopenedclosed

(In [13028]) Fixed #12286 -- Corrected order of table creation in syncdb so that it doesn't cause problems with proxied models. Thanks to everyone that helped to narrow down the problem.

comment:24 Changed 5 years ago by Jacob

milestone: 1.2

Milestone 1.2 deleted

comment:11 Changed 5 years ago by Aymeric Augustin

In [17712]:

Added cleanup code to the proxy_model_inheritance tests. Refs #12286, #16329.

comment:12 Changed 5 years ago by Aymeric Augustin

Easy pickings: unset
Severity: Normal
Type: Uncategorized
UI/UX: unset

In case someone wants to improve these tests later -- there's probably a better way -- I'm attaching a patch that re-introduces the original bug and cases these tests to fail.

Note: See TracTickets for help on using tickets.
Back to Top