Long names are generated differently in Django 1.1 and Django 1.2, causing incompatability
|Reported by:||yngve@…||Owned by:||nobody|
|Has patch:||no||Needs documentation:||no|
|Needs tests:||no||Patch needs improvement:||no|
Using long class names and member names in models can create composite table and sequence names that are longer than 63 characters, particularly for ManyToMany relations in Postgresql (v8.3)
When such a model that has been created in Django 1.1 is used in a Django 1.2 such names, particularly the id_seq, but probably others, are not found by v1.2 because it constructs a name different from what Django 1.1 originally created. I have also seen indications that this will break for models created entirely in Django 1.2, as well.
The attached testcase performs 4 different tests, the first three works in both v1.1 and v1.2, the fourth only works in v1.1.
It is possible that this problem is related to what http://code.djangoproject.com/changeset/13328 fixes, and probably related to the change to limit names to 63 characters.
I would recommend that Django at the very least should be able to use models that have been instantiated in an older version, and the names used there. Using the newer name convention should be done when 1.2 creates the tables.
For reference here are the emails I posted to django-users (At present I have downgraded the v1.2.1 systems back to v1.1.1) :
By accident I have ended up with a mixed Django 1.1.1 and Django 1.2.1 environment, on different machines.
I am currently considering whether to downgrade the 1.2 installations, or upgrade the older ones, or keep the current setup, but this depends on finding a solution to an apparent incompatibility between 1.1 and 1.2, and whether or not the upgrade can cause other problems.
The current applications, including the configuration of the Postgresql 8.3, have all been developed in v1.1.1
While this have not caused problems for one of my applications (AFAICT), I've discovered that v1.2 is apparently not compatible with the ManyToMany relations used in a second application, causing an exception to be thrown when adding to the many to many field:
File "/usr/lib/pymodules/python2.5/django/db/models/fields/related.py", line 490, in add self._add_items(self.source_field_name, self.target_field_name, *objs) File "/usr/lib/pymodules/python2.5/django/db/models/fields/related.py", line 574, in _add_items '%s_id' % target_field_name: obj_id, File "/usr/lib/pymodules/python2.5/django/db/models/query.py", line 352, in create obj.save(force_insert=True, using=self.db) File "/usr/lib/pymodules/python2.5/django/db/models/base.py", line 435, in save self.save_base(using=using, force_insert=force_insert, force_update=force_update) File "/usr/lib/pymodules/python2.5/django/db/models/base.py", line 528, in save_base result = manager._insert(values, return_id=update_pk, using=using) File "/usr/lib/pymodules/python2.5/django/db/models/manager.py", line 195, in _insert return insert_query(self.model, values, **kwargs) File "/usr/lib/pymodules/python2.5/django/db/models/query.py", line 1479, in insert_query return query.get_compiler(using=using).execute_sql(return_id) File "/usr/lib/pymodules/python2.5/django/db/models/sql/compiler.py", line 789, in execute_sql self.query.model._meta.db_table, self.query.model._meta.pk.column) File "/usr/lib/pymodules/python2.5/django/db/backends/postgresql/operations.py", line 57, in last_insert_id cursor.execute("SELECT CURRVAL('\"%s_%s_seq\"')" % (table_name, pk_name)) File "/usr/lib/pymodules/python2.5/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute return self.cursor.execute(query, args) django.db.utils.DatabaseError: relation "model_bar_foo_id_s" does not exist
A condensed example of my code (not tested independently) is the following:
class foo(models.Model): #declarations class bar(models.Model): foo_list = models.ManyToManyField(foo, null=True) def update_foo(x): self.foo_list.add(x)
The call to "self.foo_list.add()" is what triggers the above exception.
AFAICT the SQL code generated by sqlall by v1.1.1 is equivalent to the code generated by 1.2.1 (in v1.2.1 there is a use of ALTER in the bar-reference of the many-to-many table). There is however no mention in the sqlall output of an identifier with the name reported by the exception, though it is possible that it may be an identifier generated server-side, or that it would be generated in a resetdb situation.
I have not been able to discover any mention of such a problem in the v1.2 release notes.
While there will be no significant problem replacing this particular application's set of tables after an upgrade, since the application is currently not in production and I had just wiped the tables anyway, I am concerned that there may be issues in the currently working application that I have not yet discovered.
Is there a setting in v1.2 that I can use to tell the database handler that the tables were generated by v1.1? Or is there another reason for this problem?
- the psycopg2 version on the 1.1 installation is 2.0.13 (windows and Linux), the one on the 1.2 installation is 2.0.14 (linux)
- the names of the classes are much longer than used here (63 characters for the id_s name).
- There are many more ManyToMany uses in the model used by this application, including the class implicated in the exception.
This seems to have to do with long names on ManyToMany members, when model_bar_foo_id_seq exceeds 63 characters, there is also a similar problem if the referenced class name is too long.
I have now created a small testcase.
When running the foobar_test.py script after doing syncdb from Django 1.1.1 all four tests works in Django 1.1.1, while the fourth fails in 1.2.1 as mentioned below.
Then, when deleting the tables and syncdb-ing from Django 1.2.1 the fourth test still does not work, while the tests still works in 1.1.1.
Therefore, this seems to be a bug in Django 1.2.1.
There is some complaint when doing syncdb, but they seem to be for some other database operation, not configuring the tables.
It is possible that http://code.djangoproject.com/changeset/13328 may have fixed part of the problem, but when testing the last_insert_id() part of that fix, it did not work with names that contain upper case characters. I have not tested the full fix.