Opened 9 years ago

Closed 6 months ago

#2225 closed defect (fixed)

'manage.py sql ...' gets confused when using tables from separate apps

Reported by: Lucas Hazel Owned by: glassresistor
Component: Database layer (models, ORM) Version: master
Severity: minor Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by adrian)

An error message is given when using tables from a separate app if the table doesn't previously exist.

For example:

# project.app_a.models
class A(models.Model):
    # ....

# project.app_b.models
from project.app_a.models import A
class B(models.Model):
    a = models.ForeignKey(A)
    # ....

Then run python manage.py sql and you will recieve a comment claiming table app_a_a can not be referenced as it does not exist, yet the output clearly shows the table begin created.

However if one syncs the db, the tables are created and referenced despite the error message. This may be confusing to developers.

Attachments (2)

sql_patch (12.3 KB) - added by glassresistor 5 years ago.
Patch which addes a new BaseCommand based off of AppCommand called CrossAppCommand which allows multiple applications transparent access to the models of other app and matches sql printout to syncdb, fix is extended to the other sql management commands
sql_patch.diff (12.3 KB) - added by glassresistor 5 years ago.
Patch which addes a new BaseCommand? based off of AppCommand? called CrossAppCommand? which allows multiple applications transparent access to the models of other app and matches sql printout to syncdb, fix is extended to the other sql management commands

Download all attachments as: .zip

Change History (14)

comment:1 Changed 9 years ago by adrian

  • Description modified (diff)
  • Summary changed from 'manage.py sql ...' gets confused when using tables from seperate apps to 'manage.py sql ...' gets confused when using tables from separate apps

comment:2 Changed 8 years ago by SmileyChris

Looks valid, but could do with some more investigation...

comment:3 Changed 8 years ago by Gary Wilson <gary.wilson@…>

  • Triage Stage changed from Unreviewed to Accepted

comment:4 Changed 5 years ago by glassresistor@…

  • Owner changed from nobody to anonymous
  • Status changed from new to assigned

Came across this today looking for an easy bug fix. I can confirm that the error still exists in the current trunk, that syncdb does work even though sql implies that it will not work. Below I have the output for 3 models in 2 apps the foreign key inside model C has no errors but model B which is in a different app isn't aware of table app1_a's existance.

Should be able to copy how reference checking works inside apps and extend it to check other apps as well within the sql management command in the same way syncdb does this, unless its just coincidental that app1_a gets created before app2_b and then syncdb will also need some reworking.

(svn)mike@vsolanis:~/django/testtables$ python ./manage.py sql app1 app2
BEGIN;
CREATE TABLE "app1_a" (
    "id" serial NOT NULL PRIMARY KEY,
    "article_body" text NOT NULL
)
;
CREATE TABLE "app1_c" (
    "id" serial NOT NULL PRIMARY KEY,
    "key_id" integer NOT NULL REFERENCES "app1_a" ("id") DEFERRABLE INITIALLY DEFERRED
)
;
CREATE TABLE "app2_b" (
    "id" serial NOT NULL PRIMARY KEY,
    "a_id" integer NOT NULL
)
;
-- The following references should be added but depend on non-existent tables:
-- ALTER TABLE "app2_b" ADD CONSTRAINT "a_id_refs_id_ea2536ff" FOREIGN KEY ("a_id") REFERENCES "app1_a" ("id") DEFERRABLE INITIALLY DEFERRED;
COMMIT;

comment:5 Changed 5 years ago by anonymous

  • Owner changed from anonymous to glassresistor
  • Status changed from assigned to new

comment:6 Changed 5 years ago by glassresistor

After looked through the logs from a postgres 8.4 database I found that the actual queries used to create the 3 tables above through syncdb does not include the alter statement but instead is just a part of the create statement. Syncdb seems to check the full project space while sql, sqlall, sqlreset, since they take in apps as arguments loop through the listed apps and inside that, loops through models internal to the apps themselves. I propose that sqls behaviour be changed such that the models list get concatenated such that sql would in the case of sql app2 spit out -- The following references should be added but depend on non-existent tables:
but sql app1 app2 would spit out Create table statements without mentioning an alter which doesn't get done.

618-2009-12-13 10:27:27 EST LOG:  statement: CREATE TABLE "app1_a" (
619-        "id" serial NOT NULL PRIMARY KEY,
620-        "article_body" text NOT NULL
621-    )
622-    ;
623-2009-12-13 10:27:27 EST LOG:  statement: CREATE TABLE "app1_c" (
624-        "id" serial NOT NULL PRIMARY KEY,
625-        "key_id" integer NOT NULL REFERENCES "app1_a" ("id") DEFERRABLE INITIALLY DEFERRED
626-    )
627-    ;
628:2009-12-13 10:27:27 EST LOG:  statement: CREATE TABLE "app2_b" (
629-        "id" serial NOT NULL PRIMARY KEY,
630-        "a_id" integer NOT NULL REFERENCES "app1_a" ("id") DEFERRABLE INITIALLY DEFERRED
631-    )
632-    ;

Changed 5 years ago by glassresistor

Patch which addes a new BaseCommand based off of AppCommand called CrossAppCommand which allows multiple applications transparent access to the models of other app and matches sql printout to syncdb, fix is extended to the other sql management commands

comment:7 Changed 5 years ago by glassresistor

  • Has patch set
  • Needs documentation set
  • Needs tests set
  • Owner changed from glassresistor to nobody
  • Patch needs improvement set

To fix this problem I had to create a new BaseCommand extension so that unlike AppCommand CrossAppCommand looks at all of the models in the project so the table references across multiple apps will have sane outputs which can be piped into an sql database.

Changed 5 years ago by glassresistor

Patch which addes a new BaseCommand? based off of AppCommand? called CrossAppCommand? which allows multiple applications transparent access to the models of other app and matches sql printout to syncdb, fix is extended to the other sql management commands

comment:8 Changed 4 years ago by glassresistor

  • Owner changed from nobody to glassresistor
  • Patch needs improvement unset

comment:9 Changed 4 years ago by glassresistor

  • Needs documentation unset

comment:10 follow-up: Changed 4 years ago by russellm

  • Resolution set to fixed
  • Status changed from new to closed

I'm pretty sure this problem doesn't exist anymore.

comment:11 in reply to: ↑ 10 Changed 7 months ago by domonique.carter@…

  • Easy pickings unset
  • Resolution fixed deleted
  • Status changed from closed to new
  • UI/UX unset

Replying to russellm:

I'm pretty sure this problem doesn't exist anymore.

I am actually still getting the error from sqlall. I type in python manage.py sqlall data user. data and user are both applications. One of the models in user references a model in data. From my understanding of the previous statements; syncdb actually just adds the alter statement to the create statement. Is that a correct interpretation.

comment:12 Changed 6 months ago by aaugustin

  • Resolution set to fixed
  • Status changed from new to closed

I've looked at the patches proposed to fix this ticket. They are not needed after the app-loading refactor. See 6b172a6d6dcfe88be4dc0e5052707a756c1c830c and follow-up commits that touched django/core/management/__init__.py.

If you're still seeing this problem, please check that it happens with Django 1.7 (RC3 at this time) and open a new ticket -- that will be more convenient than reopening a 4 years old ticket. Thanks!

Note: See TracTickets for help on using tickets.
Back to Top