Opened 3 months ago

Closed 3 months ago

Last modified 7 weeks ago

#30033 closed Cleanup/optimization (fixed)

SQLite schema editor should use the documented process to emulate table alterations on SQlite3.

Reported by: Simon Charette Owned by: Simon Charette
Component: Migrations Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Simon Charette)

In the light of #29182 SQLite's main author mentioned that we should conform to the documented procedure to emulate table alterations on SQlite3.

Right now the SQLite's schema editor follows a procedure documented as incorrect and prevents usages of the SQLite 3.26 improvements regarding table and column renames while foreign key constraints are disabled.

Change History (12)

comment:1 Changed 3 months ago by Simon Charette

Description: modified (diff)

comment:2 Changed 3 months ago by Simon Charette

Description: modified (diff)
Has patch: set

comment:3 Changed 3 months ago by Carlton Gibson

Summary: SQLite schema editor should use the documented to emulate table alterations on SQlite3.SQLite schema editor should use the documented process to emulate table alterations on SQlite3.
Triage Stage: AcceptedReady for checkin

comment:4 Changed 3 months ago by Carlton Gibson <carlton.gibson@…>

Resolution: fixed
Status: assignedclosed

In 7289874a:

Fixed #30033 -- Conformed to the recommended table alterations procedure on SQlite3.

Refs #29182.

The previous implementation was following a procedure explicitly documented
as incorrect and was the origin of the breakage experienced on SQLite 3.26
release that were addressed by c8ffdbe514b55ff5c9a2b8cb8bbdf2d3978c188f.

Thanks to Richard Hipp for pointing out the usage of the incorrect procedure.

comment:5 Changed 3 months ago by Carlton Gibson <carlton.gibson@…>

In 894cb13:

Refs #29182 -- Stopped relying on legacy alter table semantic on SQLite 3.26+.

SQLite 3.26 changed the behavior of table and column renaming operations to
repoint foreign key references even if foreign key checks are disabled.

This makes the workarounds in place to simulate this behavior unnecessary on
SQLite 3.26+. Refs #30033.

comment:6 Changed 3 months ago by Tim Graham <timograham@…>

In 790d108c:

Refs #30033 -- Checked constraints before committing SQLite schema changes.

This order of operations is more in line with SQLite's documented table
rebuild procedure and ensures that changes aren't committed if foreign key
integrity is broken.

comment:7 Changed 3 months ago by Tim Graham <timograham@…>

In ce8b65a:

Fixed #30054 -- Implemented cascaded flush on SQLite.

This is required to maintain foreign key integrity when using
TransactionTestCase.available_apps.

Refs #30033, #14204, #20483.

comment:8 Changed 3 months ago by Tim Graham

Has patch: unset
Resolution: fixed
Severity: NormalRelease blocker
Status: closednew
Triage Stage: Ready for checkinAccepted

On Ubuntu 16.04 which has SQLite 3.11, some problems appeared after 7289874adceec46b5367ec3157cdd10c711253a0:

======================================================================
ERROR: test_m2m_rename_field_in_target_model (schema.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/media/sf_django/django/db/backends/utils.py", line 82, in _execute
    return self.cursor.execute(sql)
  File "/media/sf_django/django/db/backends/sqlite3/base.py", line 360, in execute
    return Database.Cursor.execute(self, query)
sqlite3.OperationalError: no such table: schema_tagm2mtest

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/media/sf_django/tests/schema/tests.py", line 70, in tearDown
    self.delete_tables()
  File "/media/sf_django/tests/schema/tests.py", line 97, in delete_tables
    connection.enable_constraint_checking()
  File "/media/sf_django/django/db/backends/sqlite3/schema.py", line 34, in __exit__
    self.connection.check_constraints()
  File "/media/sf_django/django/db/backends/sqlite3/base.py", line 320, in check_constraints
    column_name, referenced_column_name,
  File "/media/sf_django/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/media/sf_django/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/media/sf_django/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/media/sf_django/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/media/sf_django/django/db/backends/utils.py", line 82, in _execute
    return self.cursor.execute(sql)
  File "/media/sf_django/django/db/backends/sqlite3/base.py", line 360, in execute
    return Database.Cursor.execute(self, query)
django.db.utils.OperationalError: no such table: schema_tagm2mtest

comment:9 Changed 3 months ago by Tim Graham <timograham@…>

In 0b54ab06:

Refs #30033 -- Fixed schema's test_m2m_rename_field_in_target_model test failure on SQLite < 3.20.

Mixing local test models with non-local models resulted in a referential
integrity error during tear down since the models are removed in separate
schema editor instances which each check constraints.

Failure appeared after 7289874adceec46b5367ec3157cdd10c711253a0.

comment:10 Changed 3 months ago by Tim Graham

Resolution: fixed
Severity: Release blockerNormal
Status: newclosed

comment:11 Changed 7 weeks ago by Tim Graham <timograham@…>

In fcfb7306:

Refs #30033 -- Doc'd change regarding apps without migrations depending on apps with migrations.

The addition of self.connection.check_constraints() in
7289874adceec46b5367ec3157cdd10c711253a0 is the cause.

comment:12 Changed 7 weeks ago by Tim Graham <timograham@…>

In e8cb0dcc:

[2.2.x] Refs #30033 -- Doc'd change regarding apps without migrations depending on apps with migrations.

The addition of self.connection.check_constraints() in
7289874adceec46b5367ec3157cdd10c711253a0 is the cause.

Backport of fcfb7306586184f08c7f75c174f95b54a212320d from master.

Note: See TracTickets for help on using tickets.
Back to Top