Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#30033 closed Cleanup/optimization (fixed)

SQLite schema editor should use the documented process to emulate table alterations on SQlite3.

Reported by: Simon Charette Owned by: Simon Charette
Component: Migrations Version: dev
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Simon Charette)

In the light of #29182 SQLite's main author mentioned that we should conform to the documented procedure to emulate table alterations on SQlite3.

Right now the SQLite's schema editor follows a procedure documented as incorrect and prevents usages of the SQLite 3.26 improvements regarding table and column renames while foreign key constraints are disabled.

Change History (12)

comment:1 by Simon Charette, 5 years ago

Description: modified (diff)

comment:2 by Simon Charette, 5 years ago

Description: modified (diff)
Has patch: set

comment:3 by Carlton Gibson, 5 years ago

Summary: SQLite schema editor should use the documented to emulate table alterations on SQlite3.SQLite schema editor should use the documented process to emulate table alterations on SQlite3.
Triage Stage: AcceptedReady for checkin

comment:4 by Carlton Gibson <carlton.gibson@…>, 5 years ago

Resolution: fixed
Status: assignedclosed

In 7289874a:

Fixed #30033 -- Conformed to the recommended table alterations procedure on SQlite3.

Refs #29182.

The previous implementation was following a procedure explicitly documented
as incorrect and was the origin of the breakage experienced on SQLite 3.26
release that were addressed by c8ffdbe514b55ff5c9a2b8cb8bbdf2d3978c188f.

Thanks to Richard Hipp for pointing out the usage of the incorrect procedure.

comment:5 by Carlton Gibson <carlton.gibson@…>, 5 years ago

In 894cb13:

Refs #29182 -- Stopped relying on legacy alter table semantic on SQLite 3.26+.

SQLite 3.26 changed the behavior of table and column renaming operations to
repoint foreign key references even if foreign key checks are disabled.

This makes the workarounds in place to simulate this behavior unnecessary on
SQLite 3.26+. Refs #30033.

comment:6 by Tim Graham <timograham@…>, 5 years ago

In 790d108c:

Refs #30033 -- Checked constraints before committing SQLite schema changes.

This order of operations is more in line with SQLite's documented table
rebuild procedure and ensures that changes aren't committed if foreign key
integrity is broken.

comment:7 by Tim Graham <timograham@…>, 5 years ago

In ce8b65a:

Fixed #30054 -- Implemented cascaded flush on SQLite.

This is required to maintain foreign key integrity when using
TransactionTestCase.available_apps.

Refs #30033, #14204, #20483.

comment:8 by Tim Graham, 5 years ago

Has patch: unset
Resolution: fixed
Severity: NormalRelease blocker
Status: closednew
Triage Stage: Ready for checkinAccepted

On Ubuntu 16.04 which has SQLite 3.11, some problems appeared after 7289874adceec46b5367ec3157cdd10c711253a0:

======================================================================
ERROR: test_m2m_rename_field_in_target_model (schema.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/media/sf_django/django/db/backends/utils.py", line 82, in _execute
    return self.cursor.execute(sql)
  File "/media/sf_django/django/db/backends/sqlite3/base.py", line 360, in execute
    return Database.Cursor.execute(self, query)
sqlite3.OperationalError: no such table: schema_tagm2mtest

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/media/sf_django/tests/schema/tests.py", line 70, in tearDown
    self.delete_tables()
  File "/media/sf_django/tests/schema/tests.py", line 97, in delete_tables
    connection.enable_constraint_checking()
  File "/media/sf_django/django/db/backends/sqlite3/schema.py", line 34, in __exit__
    self.connection.check_constraints()
  File "/media/sf_django/django/db/backends/sqlite3/base.py", line 320, in check_constraints
    column_name, referenced_column_name,
  File "/media/sf_django/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/media/sf_django/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/media/sf_django/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/media/sf_django/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/media/sf_django/django/db/backends/utils.py", line 82, in _execute
    return self.cursor.execute(sql)
  File "/media/sf_django/django/db/backends/sqlite3/base.py", line 360, in execute
    return Database.Cursor.execute(self, query)
django.db.utils.OperationalError: no such table: schema_tagm2mtest

comment:9 by Tim Graham <timograham@…>, 5 years ago

In 0b54ab06:

Refs #30033 -- Fixed schema's test_m2m_rename_field_in_target_model test failure on SQLite < 3.20.

Mixing local test models with non-local models resulted in a referential
integrity error during tear down since the models are removed in separate
schema editor instances which each check constraints.

Failure appeared after 7289874adceec46b5367ec3157cdd10c711253a0.

comment:10 by Tim Graham, 5 years ago

Resolution: fixed
Severity: Release blockerNormal
Status: newclosed

comment:11 by Tim Graham <timograham@…>, 5 years ago

In fcfb7306:

Refs #30033 -- Doc'd change regarding apps without migrations depending on apps with migrations.

The addition of self.connection.check_constraints() in
7289874adceec46b5367ec3157cdd10c711253a0 is the cause.

comment:12 by Tim Graham <timograham@…>, 5 years ago

In e8cb0dcc:

[2.2.x] Refs #30033 -- Doc'd change regarding apps without migrations depending on apps with migrations.

The addition of self.connection.check_constraints() in
7289874adceec46b5367ec3157cdd10c711253a0 is the cause.

Backport of fcfb7306586184f08c7f75c174f95b54a212320d from master.

Note: See TracTickets for help on using tickets.
Back to Top