Code

Opened 4 years ago

Closed 4 years ago

Last modified 2 years ago

#12286 closed Uncategorized (fixed)

Table for Model that has been proxied may not be created

Reported by: telenieko Owned by: nobody
Component: Core (Management commands) Version: 1.1
Severity: Normal Keywords:
Cc: rvdrijst Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

Hi there,

Imagine you have

app1/models.py::

    from app2.models import NiceModel
    class ProxyModel(NiceModel):
        class Meta:
            proxy = True

app2/models.py::

   from django.db import models

   class NiceModel(models.Model):
       pass

With this code, "manage.py syncdb" will likely first "see" 'ProxyModel'. In [source:django/trunk/django/core/management/commands/syncdb.py#11780 syncdb.py]:

  • It will 'try' to (line 70) create a table for it, without doing anything.
  • But adding "app2_nicemodel" (line 82) to the list of tables "it has taken care of"
  • For that, when syncdb passes throught NiceModel it won't create any table as 'app2_nicemodel' (line 66) appears as created.

To reproduce that you need syncdb to be taking into account the "proxy" models before the realones (may happen when the "proxying" app is alphabetically before the real one).

Attachments (5)

proxy_model_inheritance_test.diff (2.7 KB) - added by flyingfred0 4 years ago.
syncdb_proxy_model.diff (791 bytes) - added by flyingfred0 4 years ago.
Patch to fix 1.1.X branch
12286-r12948.diff (681 bytes) - added by ramiro 4 years ago.
t12286.diff (1.5 KB) - added by russellm 4 years ago.
Possible fix for syncdb problems with proxy models?
16329-apply-this-to-reproduce-proxy-model-inheritance-failure.diff (2.2 KB) - added by aaugustin 2 years ago.

Download all attachments as: .zip

Change History (31)

comment:1 Changed 4 years ago by russellm

  • milestone set to 1.2
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

comment:2 Changed 4 years ago by flyingfred0

  • Owner changed from nobody to flyingfred0
  • Status changed from new to assigned

Changed 4 years ago by flyingfred0

comment:3 Changed 4 years ago by flyingfred0

  • Has patch set

This bug appears to have been fixed by [12597]. The attached patch is simply an additional test that reproduces the issue prior to [12597].

Changed 4 years ago by flyingfred0

Patch to fix 1.1.X branch

comment:4 Changed 4 years ago by flyingfred0

  • Owner changed from flyingfred0 to nobody
  • Status changed from assigned to new

The proxy_model_inheritance_test patch is valid for both trunk and the 1.1.X branch. The syncdb_proxy_model patch fixes this issue for the 1.1.X branch.

comment:5 Changed 4 years ago by kmtracey

  • Resolution set to fixed
  • Status changed from new to closed

(In [12829]) [1.1.X] Fixed #12286: Ensure proxied model's table is created.
Thanks to telenieko for the report and flyingfred0 for test and fix.

Test is r12828 from trunk.

comment:6 Changed 4 years ago by tomxtobin

  • Resolution fixed deleted
  • Status changed from closed to reopened

I'm seeing the new proxy_model_inheritance test added in r12828 raise an exception under PostgreSQL (backend postgresql_psycopg2):

======================================================================
ERROR: test_table_exists (modeltests.proxy_model_inheritance.tests.ProxyModelInheritanceTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ttobin/repo/django/tests/modeltests/proxy_model_inheritance/tests.py", line 35, in test_table_exists
    self.assertEquals(NiceModel.objects.all().count(), 0)
  File "/Users/ttobin/lib/python2/packages/django/db/models/query.py", line 324, in count
    return self.query.get_count(using=self.db)
  File "/Users/ttobin/lib/python2/packages/django/db/models/sql/query.py", line 368, in get_count
    number = obj.get_aggregation(using=using)[None]
  File "/Users/ttobin/lib/python2/packages/django/db/models/sql/query.py", line 340, in get_aggregation
    result = query.get_compiler(using).execute_sql(SINGLE)
  File "/Users/ttobin/lib/python2/packages/django/db/models/sql/compiler.py", line 729, in execute_sql
    cursor.execute(sql, params)
  File "/Users/ttobin/lib/python2/packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
    return self.cursor.execute(query, args)
DatabaseError: relation "app2_nicemodel" does not exist
LINE 1: SELECT COUNT(*) FROM "app2_nicemodel"

comment:7 Changed 4 years ago by kmtracey

Odd. Test passes when run with sqlite, MySQL (both InnoDB and MyISAM), and Oracle. Fails with PostgreSQL, on both trunk and the 1.1.X branch. I've no idea why.

comment:8 Changed 4 years ago by russellm

  • Resolution set to fixed
  • Status changed from reopened to closed

(In [12846]) Fixed #12286 -- Modified the test case added in r12828 so that it passes under Postgres.

comment:9 Changed 4 years ago by ramiro

  • Component changed from Database layer (models, ORM) to django-admin.py
  • milestone 1.2 deleted
  • Resolution fixed deleted
  • Status changed from closed to reopened

While debugging another unrelated issue I found that the situation described by the OP is still happening:

When the 'app1' app (the one containing the proxy model) from the 'proxy_model_inheritance' tests added to fix this ticket is processed in this loop, the "app2_nicemodel" table corresponding to the 'app2.models.NiceModel' is being reported as part of this app's tables manifest.

Later, when the 'app2' application is processed the same table is reported again and blindly added again to the 'tables' collection (that is implemented with a list).

I'm reopening because this is a code correctness fix, I'm dropping the 1.2 milestone though.

Changed 4 years ago by ramiro

comment:10 Changed 4 years ago by rvdrijst

  • Cc rvdrijst added

comment:11 follow-up: Changed 4 years ago by jcassee

We encounter this problem during syncdb (and by extension test) of our project using Django trunk (r12970); the project has become unusable with Postgresql. The latest patch by ramiro (12286-r12948.diff) does not fix it for us. If there is anything we can do to help debugging the problem, please let up know.

comment:12 Changed 4 years ago by russellm

  • milestone set to 1.2
  • Patch needs improvement set

If there is a still a regressing behavior, that needs to be addressed before 1.2

comment:13 in reply to: ↑ 11 ; follow-up: Changed 4 years ago by ramiro

Replying to jcassee:

We encounter this problem during syncdb (and by extension test) of our project using Django trunk (r12970); the project has become unusable with Postgresql. The latest patch by ramiro (12286-r12948.diff) does not fix it for us. If there is anything we can do to help debugging the problem, please let up know.

Isolation of a reduced test case that shows the problem you are experiencing and posting it here would be very helpful.

comment:14 in reply to: ↑ 13 Changed 4 years ago by jcassee

Replying to ramiro:

Isolation of a reduced test case that shows the problem you are experiencing and posting it here would be very helpful.

I would, but the test runner calls syncdb and the exception is thrown. So it would have to be somthing with a reduced INSTALLED_APPLICATIONS list. I may try the above snippets, but I have a suspicion that it will pass the test. I will try anyway and report back.

comment:15 Changed 4 years ago by jcassee

Running the proxy_model_inheritance test passes after cleaning out the INSTALLED_APPS list. Isolating a reduced test case will take some time, as the package that contains the proxy models depends on a number of apps.

comment:16 Changed 4 years ago by russellm

Unfortunately, unless someone can provide a test case to verify that this problem still exists, we'll need to drop this ticket from the release blocker list for 1.2. If you can provide *any* solid evidence that this problem still exists, we need to see it very soon.

comment:17 Changed 4 years ago by harm

We see this problem too. I can confirm its not fixed.

I've however not (yet) been able to reproduce the issue with a cleaned out INSTALLED_APPS. When trying then all just seems to work. The ordering of of table creation seems to influence this bug.

comment:18 Changed 4 years ago by harm

here is a testcase:

settings.py

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.messages',

    'tagging',
    'anapp',

)

where tagging is django-tagging. (http://code.google.com/p/django-tagging/)

an anapp is an almost empty app with with the following models

from django.db import models

from tagging.models import Tag, TaggedItem

class TagProxy(Tag):
    class Meta:
        proxy = True

class TaggedItemProxy(TaggedItem):
    class Meta:
        proxy = True

Now when doing a manage.py syncdb or manage.py test the Tag & TaggedItem tables are _not_ created.

(python2.5)harm@dev2:~/svn/idq/trunk/projects/idqsso/django/projects/ep_auth_demo$ python manage.py test
Creating test database 'default'...
Creating table django_session
Creating table django_site
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_user_permissions
Creating table auth_user_groups
Creating table auth_user
Creating table auth_message
Creating table django_content_type
Installing index for auth.Permission model      <<<<<<<<<<<<<<<< before here the tagging tables should be created

Changed 4 years ago by russellm

Possible fix for syncdb problems with proxy models?

comment:19 follow-up: Changed 4 years ago by russellm

@harm: I can't reproduce - using that example, I see the tagging tables synchronized, both under syncdb and test, exactly where you indicate that they aren't.

However, looking at syncdb, it occurs to me that the problem could be one of dictionary ordering. Can you try the patch I just uploaded to see if it fixes the problem for you?

comment:20 Changed 4 years ago by harm

I can no longer reproduce the problem on trunk revision [13027].
(NB: without your patch: t12286.diff ).

(seen on [12403] and a unknown trunk version of around 1.5 weeks ago)

Do others still see the issue ? (jcassee, ramiro)

comment:21 Changed 4 years ago by marcusf

[12590] Is the last revision where I can reproduce the problem. Seems like [12597] fixed it.

comment:22 in reply to: ↑ 19 Changed 4 years ago by jcassee

Replying to russellm:

However, looking at syncdb, it occurs to me that the problem could be one of dictionary ordering. Can you try the patch I just uploaded to see if it fixes the problem for you?

Yes, your patch fixes the problem for us. Thanks a lot, Russell! (Note that without the patch syncdb still fails, even on r12970.)

comment:23 Changed 4 years ago by russellm

  • Resolution set to fixed
  • Status changed from reopened to closed

(In [13028]) Fixed #12286 -- Corrected order of table creation in syncdb so that it doesn't cause problems with proxied models. Thanks to everyone that helped to narrow down the problem.

comment:24 Changed 3 years ago by jacob

  • milestone 1.2 deleted

Milestone 1.2 deleted

comment:11 follow-up: Changed 2 years ago by aaugustin

In [17712]:

Added cleanup code to the proxy_model_inheritance tests. Refs #12286, #16329.

comment:12 Changed 2 years ago by aaugustin

  • Easy pickings unset
  • Severity set to Normal
  • Type set to Uncategorized
  • UI/UX unset

In case someone wants to improve these tests later -- there's probably a better way -- I'm attaching a patch that re-introduces the original bug and cases these tests to fail.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.